The links are buried somewhere on Phoronix, I am looking... But what I am saying is Google's rejection of JXL seems to be as bad as it looks.
7,874 karma · joined February 13, 2023
The links are buried somewhere on Phoronix, I am looking... But what I am saying is Google's rejection of JXL seems to be as bad as it looks.
What I'm saying is that whatever engineering and environmental limitations we currently perceive are probably irrelevant.
Not being multiplanetary seems like the least of our existential problems here on Earth, and will continue to be that way for awhile.
At the same time, chemical rocket efficiency becomes totally irrelevant for a slightly more advanced civilization than us.
Even an analytical article on the Musk cult can't escape the Musk cult. Its just too profitable.
Discord is so time inefficient, its almost hilarious. For every incredible conversation between experts you observe, you have to weed through 200 times as much filler.
It will not be, as long as it is the internet gatekeepers' livlihood.
You just said it!
First of all, engagement optimization is as old as the printing press.
...But the difference now is thay is that news targets algorithms, not eyeballs and reputation.
If you are talking about transformers llms that output pixels, that is an area of active research... I hope you have a few million dollars lying around, though.
I am the biggest local ML advocate you will find. My 3090 is either running Yi 34B queries or other experiments all day, my job is with local LLMS... But I am totally OK with heavy handed AI-related moderation. I dont want the sea of AI grifters to have a single second on the HN front page.
RADV even has big commercial customers now (Valve).
Slap taxes above a certain weight, and you will see manufacturers drop the weight to right below the threshold faster than you can blink.
I suspect the "race edition" stripped version of many cars are still hauling around the fundamental design features they need to accommodate luxury features.
AFAIK the limiting factors are tire grip, ABS control and of course mass.
Modern cars are just so huge, and have so much junk in them. I get a large fraction of the mass is for safety, but (for the same money) I'd much rather have a tighter, lighter, stripped down car than a clunky infotainment system, seat coolers, fragile wood trim, motorized ac vents and handles. I'll even sacrifice some sound deadening and rear AC.
And EVs, of course, just make this issue worse. Their "fuel" is such a large fraction of their mass that they more-or-less suffer from the tyranny of the rocket equation, where they need more battery to haul more stuff around, which in turn requires more battery, and now the chassis and suspension needs to be reinforced to handle the weight, and now that needs more battery and so on.
Everything is moving so fast!
Its not easy to install though, and big dGPU only.
Co writing a long story with the model so it references past plot points.
The story writing in particular is just something you can't possibly do well with RAG. You'd be surprised how well LLMS can "understand" a mega context and grasp events, implications and themes from them.
I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.
However, the llama.cpp server is... very buggy. The OpenAI endpoint doesn't work. It hangs and crashes constantly. I don't see how anyone could use it for batched production as of last november/december.
The reason I don't use llama.cpp personally is no flash attention (yet) and no 8 bit kv cache, so its not too great at long (32K+) contexts. But this is a niche, and being addressed.
TBH I do not follow OpenAI much. I like my personal models local, and my workplace likes their models local as well.
Some are pretty good! Check out this little curated nugget: https://llm-tracker.info/
I used to follow one with a UI that resembled HN itself, but now I can't find it in my bookmarks, lol.
Whats interesting is that Mistral apparently distributed these GGUFs. This is (in my experience) not a good format for production, so I am curious exactly who was wanting to test a model in GGUF.
No one knows about it! Which is ridiculous because batched requests with loras is mind blowing! Just like many other awesome backends like InternLM's backend, LiteLLM, Outline's VLLM fork, Aphroidte, exllamav2 batching servers and and such. Heck, a lot of trainers don't even publish the loras they merge into base models.
Personally we are waiting on the integration with constrained grammar before swapping to Lorax. Then I am going to add exl2 quantization support myself... I hope.
And yeah, /r/LocalLlama seems to be getting noisier.
TBH I just follow people and discuss stuff on huggingface directly now. Its not great, but at least its not discord.
I can't say I blame them either, there is a lot of insane crypto-like fraud in the LLM/GenAI space. I watch the space like a hawk... and I couldn't even tell you how to filter it, it's a combination of self-training from experience and just downloading and testing stuff myself.
We are getting a lot of great models out of China in particular (Yi, Qwen, InternLM, ChatGLM), and some good continuations like Solar.
Lots of amazing papers on architectures and long context are coming out.
Backends are going crazy. Outlines is shoving constrained generation everywhere, and Lorax is a LLM revelation as far as I'm concerned.
But you won't hear about any of this on Twitter/HN. Pretty much the only thing people tweet about is vllm/llama.cpp and llama/mistral, but there's a lot more out there than that.
Seems like a great candiate to merge with other 70Bs as well. There aren't a lot of really great 70B training continuations, like CodeLlama or sequelbox's continuations.