HNHacker News
TopNewBestAskShowJobs

treprinum

1,234 karma · joined May 30, 2023

Done
submissionscomments
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
How about running your intelligent home with a voice assistant on your own computer? In privacy-oriented countries (Germany) that would be massive.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
My comment was about Intel having a starter project, getting enthusiastic response from devs, network effects and iterate from there. They need a way to threaten Nvidia and just focusing on what they can't do won't bring them there. There is one route where they can disturb Nvidia's high end over time and that's a cheap basic GPU with lots of RAM. Like Ryzen 1st gen whose single core performance was two generations behind Intel trashed Intel by providing 2x as many cores for cheap.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
Intel has Xeon Phi which was a spin-off of their first attempt at GPU so they have a lot of tech in place they can reuse already. They don't need to go with GDDRx/HBMx designs that require large dies.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
4080 won't do video due to low RAM. The GPU doesn't have to be as fast there, it can be 5x slower which is still way faster than a CPU. And Intel can iterate from there.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on those larger LLMs.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
Nothing in my comment says about pricing it at the M4 Max level. Apple charges as much because they can (typing this on an $8000 M3 Max). 128GB LPDDR5 is dirt cheap these days just Apple adds its premium because they like to. Nothing prevents Intel from releasing a basic GPU with that much RAM for under $1k.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
Inference on very large LLMs where model + backprop exceed 48GB is already way faster on a 128GB MacBook than on NVidia unless you have one of those monstrous Hx00s with lots of RAM which most devs don't.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
Sufficient for LLMs and image/video gen.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
NVidia and AMD make $$$ on datacenter GPUs so it makes sense they don't want to discount their own high-end. Intel has nothing there so they can happily go for commodization of AI hardware like what Meta did when releasing LLaMA to the wild.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
They can use LPDDR5x, it would still massively accelerate inference of large local LLMs that need more than 48GB RAM. Any tensor swapping between CPU RAM and GPU RAM kills the performance.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070), and using some basic RAM like LPDDR5x it would still be way faster than anything from NVidia.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
llama.cpp and its derivatives say yes.
treprinum··on Intel announces Arc B-series "Battlemage" discrete graphics with Linux support
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...
treprinum··on Understanding the BM25 full text search algorithm
text-embedding-3-large + SPLADE + RRF
treprinum··on Show HN: FastGraphRAG – Better RAG using good old PageRank
Anticipate what kind of questions user might ask, pre-compute the answers and store them as natural sentences in a vector database.
treprinum··on M4 MacBook Pro
I hoped for at least 192GB of RAM for LLMs as 48GB DDR5 are pretty normal nowadays.
treprinum··on M4 MacBook Pro
Poor. My M3 Max/128GB is about 20x slower than 4090. For inference it's much better, still much slower than 4090 but it enables working with much larger LLMs albeit at ~10t/s (in comparison, Threadripper 2990WX/256GB does like 0.25t/s). M4 Max is likely going to be ~25% faster than M3 Max based on CPU perf and memory bandwidth.
treprinum··on Eazel, ex-Apple led Linux startup
Who designed the icons in MacOS up to Mavericks and iOS 6? Those were IMO the peak icon design on Apple devices. Would be great to hire that person for some Linux UI.
treprinum··on 80% of AI Projects Crash and Burn, Billions Wasted Says Rand Report
Isn't that better than normal? It used to be 90% of startups going belly up within 3 years, and out of remaining 10%, 9% becoming zombies and 1% having a proper exit? This looks more like Pareto 80/20 which is way better.
treprinum··on Meta cancels high-end mixed reality headset after Apple Vision Pro struggles
Try The Quest: Everest next.

https://www.thequesteverest.com/

treprinum··on 17-year-old student exposes Germany's 'secret' pirate site blocklist
I can confirm, they are banned but VPN or Tor can access them without any issues. So it's only to prevent normies from accessing them.
treprinum··on I'm tired of fixing customers' AI generated code
Set firm boundaries and reject/ghost customers that want to build their app for free and are "angry". Those never lead to anything profitable.
treprinum··on IT tycoon Mike Lynch, daughter Hannah found dead
Does HP have a black magic corporate division?
treprinum··on Markov chains are funnier than LLMs
Generative AI often uses Markov chains as its theoretical basis and is modeled in the language of conditional probabilities albeit not stuck to a single ancestor but to a chain of ancestors. One can often view LLMs as scaled out Markov chains (more like "Erlang" chains allowing deeper memory) that encode the probability of the next token based on the past of the size of the context window and then doing some magic when picking next tokens.
treprinum··on WTH Is Cerebrolysin, Actually?
While the science behind it is shoddy, some people with neuro long covid report significant improvement from it. Perhaps there is some not yet unearthed mechanism making it work in some cases?
treprinum··on Kim Dotcom's extradition to the U.S. given green light by New Zealand
Didn't YouTube get popular on pirated content first? What was the main difference of the initial phase of YouTube and MegaUpload? They both went legal later.
treprinum··on AMD records its highest server market share in decades
Yeah, but the "AI" on CPU means some basic NPU doing (sparse) matrix multiplication, not AVX512.
treprinum··on AMD records its highest server market share in decades
All these points about Intel are just waiting for an answer to "Is Intel capable of changing its prevailing culture or not?". There is a reason why Intel fell behind and it wasn't due to a lack of engineering geniuses inside. You might bet on the stock market and see. AMD obviously has a SPOF in TSMC and its limited capacity.
treprinum··on AMD records its highest server market share in decades
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's still due to shared memory with GPU).
treprinum··on AMD records its highest server market share in decades
The article had a separate "mobile" category for that where AMD had <20% market share.
← PreviousPage 2 of 22Next →