My comment was about Intel having a starter project, getting enthusiastic response from devs, network effects and iterate from there. They need a way to threaten Nvidia and just focusing on what they can't do won't bring them there. There is one route where they can disturb Nvidia's high end over time and that's a cheap basic GPU with lots of RAM. Like Ryzen 1st gen whose single core performance was two generations behind Intel trashed Intel by providing 2x as many cores for cheap.
Intel has Xeon Phi which was a spin-off of their first attempt at GPU so they have a lot of tech in place they can reuse already. They don't need to go with GDDRx/HBMx designs that require large dies.
4080 won't do video due to low RAM. The GPU doesn't have to be as fast there, it can be 5x slower which is still way faster than a CPU. And Intel can iterate from there.
4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on those larger LLMs.
Nothing in my comment says about pricing it at the M4 Max level. Apple charges as much because they can (typing this on an $8000 M3 Max). 128GB LPDDR5 is dirt cheap these days just Apple adds its premium because they like to. Nothing prevents Intel from releasing a basic GPU with that much RAM for under $1k.
Inference on very large LLMs where model + backprop exceed 48GB is already way faster on a 128GB MacBook than on NVidia unless you have one of those monstrous Hx00s with lots of RAM which most devs don't.
NVidia and AMD make $$$ on datacenter GPUs so it makes sense they don't want to discount their own high-end. Intel has nothing there so they can happily go for commodization of AI hardware like what Meta did when releasing LLaMA to the wild.
They can use LPDDR5x, it would still massively accelerate inference of large local LLMs that need more than 48GB RAM. Any tensor swapping between CPU RAM and GPU RAM kills the performance.
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070), and using some basic RAM like LPDDR5x it would still be way faster than anything from NVidia.
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...
Poor. My M3 Max/128GB is about 20x slower than 4090. For inference it's much better, still much slower than 4090 but it enables working with much larger LLMs albeit at ~10t/s (in comparison, Threadripper 2990WX/256GB does like 0.25t/s). M4 Max is likely going to be ~25% faster than M3 Max based on CPU perf and memory bandwidth.
Who designed the icons in MacOS up to Mavericks and iOS 6? Those were IMO the peak icon design on Apple devices. Would be great to hire that person for some Linux UI.
Isn't that better than normal? It used to be 90% of startups going belly up within 3 years, and out of remaining 10%, 9% becoming zombies and 1% having a proper exit? This looks more like Pareto 80/20 which is way better.
Generative AI often uses Markov chains as its theoretical basis and is modeled in the language of conditional probabilities albeit not stuck to a single ancestor but to a chain of ancestors. One can often view LLMs as scaled out Markov chains (more like "Erlang" chains allowing deeper memory) that encode the probability of the next token based on the past of the size of the context window and then doing some magic when picking next tokens.
While the science behind it is shoddy, some people with neuro long covid report significant improvement from it. Perhaps there is some not yet unearthed mechanism making it work in some cases?
Didn't YouTube get popular on pirated content first? What was the main difference of the initial phase of YouTube and MegaUpload? They both went legal later.
All these points about Intel are just waiting for an answer to "Is Intel capable of changing its prevailing culture or not?". There is a reason why Intel fell behind and it wasn't due to a lack of engineering geniuses inside. You might bet on the stock market and see. AMD obviously has a SPOF in TSMC and its limited capacity.
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's still due to shared memory with GPU).