7,874 karma · joined February 13, 2023
https://huggingface.co/brucethemoose/LargeWorldModel_LWM-Tex...
https://huggingface.co/dranger003/LWM-Text-Chat-128K-iMat.GG...
And its long context recall is quite good! We've already kind of discovered this with Yi, but there are some things one can do with a mega context that you just can't get with RAG.
We've had Yi 6B 200K context for some time, which is also quite good.
The problem, of course, is hardware requirements and vram. This one is particularly hairy since its not a GQA model.
You can quantize the cache and fit quite a bit on GPUs. At least 75k on my mere 24GB 3090, maybe 200K with a fancy quantization repo.
https://huggingface.co/LargeWorldModel
And the model page specifically mention Books3
...But the climate change problem is now social, and I think its largely a symptom of the information dysfunction that is getting worse across the globe.
Llama.cpp is much slower, and does not have built-in RAG.
TRT-LLM is a finicky deployment grade framework, and TBH having it packaged into a one click install with llama index is very cool. The RAG in particular is beyond what most local LLM UIs do out-of-the-box.
ComfyUI is cool but very DIY. You don't get good results unless you wrap your head around all the augmentations and defaults.
No idea if it will support cascade.
The unusual thing seems to be how two such objects ended up in such an orbit... But that doesn't seem to be something people would upvote.
> Difficulty of explaining the current wide orbit The current separation between the luminous star and dark companion is ≈ 2.2 au. This separation is uncom- fortable because it is both too tight for the progenitor of the NS to have fit inside as a red supergiant or AGB star, and too wide to be a result of common envelope evolution.
Maybe upvoters think, based on the title, it's a near Earth supernova candidate or something? But (among other things) the compact object is too far away from it's compamion star and not close enough to the point where a neutron star would collapse. And its pretty far away.
I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any. It seems they are unobtanium for megacaps only.
But they give a lot of TPU time away for research, which is nice.
It seems Intel Gaudi 2 is priced in a sweeter spot, but I've never head of anyone but Intel using them.
40 CUs, 256 bit LPDDR5X, 16 CPU cores. Or so the rumors say.
But on the other hand the software stack is very mature and they are heavily amortized by the huge volume, so its kinda hard to argue with. In fact its so good that Nvidia can charge outrageous prices for the L40, A10 and such and then turn around and sell the exact same dies to consumers (with less memory).
Functionary is a series of function calling llama architecture finetuned models, not a new base model, if thats what you mean.
Hence, it can be turned into a gguf like any other llama finetune.
It just has a very esoteric prompting format, but llama-cpp-python is specifically set up to leverage it.
Bethesda Game Studios is getting there.
I don't know about indie games as much, but I know Distant Worlds 2 got bit by its exotic engine.
It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then its not universally compatible and that's just going to create more confusion.
Its actually great. Most backends and frontends "just work" with each other because they all talk with openai (albeit with some caveats).
There is a growing graveyard of games/devs suffering from bespoke engine woes instead of just using Unreal...
Technically a wrapper of a wrapper, though llama-cpp-python is quite excellent... TBH I would just recommend using that and all its under-the-radar features (like grammar, and built in function calling support with Functionary).
Speaking as a 7800X3D owner, I don't think any CPU on the planet is sufficient for modded Rimworld yet.
One would think its all GPU. But you'd be suprised how single-core-speed limited parts of certain tasks are.
In the US, where its kind of a possibility... I'm not sure. But I'd be pretty pissed at the parties I feel have an underlying responsibility for the conflict (like, trying to stay relevant to HN, social media platforms that profit off of polarization), so I think I'd lay low, dodge the draft and try to fight that instead.
And its done incorrectly way too often.
(I am sitting on no high horse here, I'm guilty of it as well).
I hate to claim something without a citation, but for the life of me I can't find my Phoronix comment... Only my other comments referencing it.