HNHacker News
TopNewBestAskShowJobs

brucethemoose2

7,874 karma · joined February 13, 2023

submissionscomments
brucethemoose2··on World model on million-length video and language with RingAttention
Or more specifically, be able to look at everything all at once instead of in chunks.
brucethemoose2··on Conformant OpenGL 4.6 on the M1
And Sodium! (For minecraft)
brucethemoose2··on World model on million-length video and language with RingAttention
We've been testing it in the local llm Discords, turns out its just a llama 7B finetune that can run on any old GPU (which is cool).

https://huggingface.co/brucethemoose/LargeWorldModel_LWM-Tex...

https://huggingface.co/dranger003/LWM-Text-Chat-128K-iMat.GG...

And its long context recall is quite good! We've already kind of discovered this with Yi, but there are some things one can do with a mega context that you just can't get with RAG.

brucethemoose2··on World model on million-length video and language with RingAttention
Its more of a tech demo since its llama 7B (a model that is, TBH, obsoleted by mistral 7B), and its dataset is not that great.

We've had Yi 6B 200K context for some time, which is also quite good.

The problem, of course, is hardware requirements and vram. This one is particularly hairy since its not a GQA model.

brucethemoose2··on World model on million-length video and language with RingAttention
Its llama 7B, so anything that runs that.

You can quantize the cache and fit quite a bit on GPUs. At least 75k on my mere 24GB 3090, maybe 200K with a fancy quantization repo.

brucethemoose2··on World model on million-length video and language with RingAttention
They have at least some of the dataset uploaded:

https://huggingface.co/LargeWorldModel

And the model page specifically mention Books3

brucethemoose2··on Charts about Climate Change
How very reasonable.

...But the climate change problem is now social, and I think its largely a symptom of the information dysfunction that is getting worse across the globe.

brucethemoose2··on Nvidia's Chat with RTX is an AI chatbot that runs locally on your PC
> the more technical users will leverage llama.cpp to run whatever models they are interested in.

Llama.cpp is much slower, and does not have built-in RAG.

TRT-LLM is a finicky deployment grade framework, and TBH having it packaged into a one click install with llama index is very cool. The RAG in particular is beyond what most local LLM UIs do out-of-the-box.

brucethemoose2··on Stable Cascade
Fooocus is the fastest way to try SDXL/SDXL turbo with good quality.

ComfyUI is cool but very DIY. You don't get good results unless you wrap your head around all the augmentations and defaults.

No idea if it will support cascade.

brucethemoose2··on Stable Cascade
What's more, they can presumably be swapped in and out like the SDXL base + refiner, right?
brucethemoose2··on A 1.9 solar mass neutron star candidate in a 2-year orbit
Indeed.

The unusual thing seems to be how two such objects ended up in such an orbit... But that doesn't seem to be something people would upvote.

> Difficulty of explaining the current wide orbit The current separation between the luminous star and dark companion is ≈ 2.2 au. This separation is uncom- fortable because it is both too tight for the progenitor of the NS to have fit inside as a red supergiant or AGB star, and too wide to be a result of common envelope evolution.

Maybe upvoters think, based on the title, it's a near Earth supernova candidate or something? But (among other things) the compact object is too far away from it's compamion star and not close enough to the point where a neutron star would collapse. And its pretty far away.

brucethemoose2··on A 1.9 solar mass neutron star candidate in a 2-year orbit
1.9 solar masses is not too unusual.
brucethemoose2··on Nvidia is now more valuable than Amazon and Google
There's more ROCm compatibility than you'd think.

I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any. It seems they are unobtanium for megacaps only.

brucethemoose2··on Nvidia pursues $30B custom chip opportunity with new unit
Google prices the TPUs pretty exorbitantly, actually.

But they give a lot of TPU time away for research, which is nice.

It seems Intel Gaudi 2 is priced in a sweeter spot, but I've never head of anyone but Intel using them.

brucethemoose2··on Nvidia pursues $30B custom chip opportunity with new unit
We are getting that (in early 2025?) With AMD Strix Halo.

40 CUs, 256 bit LPDDR5X, 16 CPU cores. Or so the rumors say.

brucethemoose2··on Nvidia pursues $30B custom chip opportunity with new unit
AWS/Azure prices are really awful TBH. There are already much better places to get GPUs.
brucethemoose2··on Nvidia pursues $30B custom chip opportunity with new unit
There is a lot of silicon consumer GPUs don't "need" for AI.

But on the other hand the software stack is very mature and they are heavily amortized by the huge volume, so its kinda hard to argue with. In fact its so good that Nvidia can charge outrageous prices for the L40, A10 and such and then turn around and sell the exact same dies to consumers (with less memory).

brucethemoose2··on Localllm lets you develop gen AI apps on local CPUs
This question is confusing to me.

Functionary is a series of function calling llama architecture finetuned models, not a new base model, if thats what you mean.

Hence, it can be turned into a gguf like any other llama finetune.

It just has a very esoteric prompting format, but llama-cpp-python is specifically set up to leverage it.

brucethemoose2··on Localllm lets you develop gen AI apps on local CPUs
Grammar has nothing to do with gguf, as it's logit processing (and that is being integrated into other backends as well).
brucethemoose2··on Disney to take $1.5B stake in Epic Games
Also, Mass Effect Andromeda.

Bethesda Game Studios is getting there.

I don't know about indie games as much, but I know Distant Worlds 2 got bit by its exotic engine.

brucethemoose2··on Localllm lets you develop gen AI apps on local CPUs
On the contrary, it's not one-size-fits-all.

It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then its not universally compatible and that's just going to create more confusion.

brucethemoose2··on Localllm lets you develop gen AI apps on local CPUs
Everyone uses that for openai compatible api requests.

Its actually great. Most backends and frontends "just work" with each other because they all talk with openai (albeit with some caveats).

brucethemoose2··on Disney to take $1.5B stake in Epic Games
Also, game dev is hard. Engine dev is really hard.

There is a growing graveyard of games/devs suffering from bespoke engine woes instead of just using Unreal...

brucethemoose2··on Localllm lets you develop gen AI apps on local CPUs
Even Google is making a llama.cpp wrapper.

Technically a wrapper of a wrapper, though llama-cpp-python is quite excellent... TBH I would just recommend using that and all its under-the-radar features (like grammar, and built in function calling support with Functionary).

brucethemoose2··on AMD Ryzen 5 8500G: A surprisingly fascinating sub-$200 CPU
Or trying to run large maps in sim games.

Speaking as a 7800X3D owner, I don't think any CPU on the planet is sufficient for modded Rimworld yet.

brucethemoose2··on AMD Ryzen 5 8500G: A surprisingly fascinating sub-$200 CPU
Actually... machine learning.

One would think its all GPU. But you'd be suprised how single-core-speed limited parts of certain tasks are.

brucethemoose2··on Show HN: CPU Prices on eBay
"AI" GPUs like the 3090 and 3060 are starting to shoot up in price, unfortunately.
brucethemoose2··on [dead]
I mean, it depends on the circumstances.

In the US, where its kind of a possibility... I'm not sure. But I'd be pretty pissed at the parties I feel have an underlying responsibility for the conflict (like, trying to stay relevant to HN, social media platforms that profit off of polarization), so I think I'd lay low, dodge the draft and try to fight that instead.

brucethemoose2··on Still no love for JPEG XL: Browser maker love-in snubs next-gen image format
> tone mapping is part of artistic creativity

And its done incorrectly way too often.

(I am sitting on no high horse here, I'm guilty of it as well).

brucethemoose2··on Still no love for JPEG XL: Browser maker love-in snubs next-gen image format
Hmm, wasn't the name I remembered.

I hate to claim something without a citation, but for the life of me I can't find my Phoronix comment... Only my other comments referencing it.

← PreviousPage 4 of 34Next →