Hacker News Reader

Comment by Zagitta | Hacker News Reader

Hey this is my work, although the exploits etc aren't my ideas by any means. This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.

Happy to answer questions!

View on HN

Thanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics.

great to hear, which quant + KV quant + KV size are you on? is it llama.cpp?

Here's the writeup from the experiments that claude did: https://claude.ai/artifact/X6WJVtKdL1LrhYuCMqdXvG#b9a4d3e2-8...

it's got a loginwall

I just copied and pasted it, can't do markdown.

https://pastebin.com/FAdBbGUd Qwen3.8-27B on an RTX 3090: Inference Options and Benchmark Results

Thank you!

Super well done, I wished all mods took your approach. Patched drivers are a nuisance, especially if you have more than one kind of card. Thank you.

edit: now do the 4090 ;)

Yeah I hate the patched drivers too, I have a CMP170 as well I want to see if I can find a way to get fully working the same way without or minimal driver patching.

4000s is WIP, forks has already unlocked it but I'm doing some refactoring to better support different families and after that I'll port it over :-)

Can't wait, I have two 48G modded ada 4090's and 4 stock Blackwell 4000's (though I doubt those will be as easy). The 4090's alone would be an amazing thing to have boosted, they're my main workhorses and effectively any improvement there will immediately make my days more productive. If I can help out with testing, my email is in my profile.

Blackwell and hopper have a whole new boot process so those are unlikely to fall over anytime soon from what I've seen.

If you can't wait for 4090 someone has a fork with a seemingly working unlock :D

Yes, but it's pinned to a driver version that I unfortunately can not use. That's precisely why I love your solution, it is the best way to do this.

I'm running 8 GPUs attached to a single CPU and that driver is a pretty essential part of getting this to work with P2P between as many cards as possible. The modded 4090 is a problem because the BAR1 it advertises (32G) is smaller than the VRAM so not all software is able to use it (you need two memory segments rather than a single 48G one to make this work). That driver is part of the solution for this problem.

FWIW the gpu unlock ada repo doesn't pin to a driver version, Codex for me just compiled it, and it works with my 615.xx driver even though the fork is for 610.x. I think the repo just uses the firmware from that version. My GPU is 4070ti super.

Ok, trying it. I'm using a pretty heavily modified 590.48.01 to be able to use those 48G 4090s, they work but they do require a bit of handholding. The main problem is the BAR1 just isn't large enough, it's capped at 32G.

Heh, got it working, thank you so much!!

Overall 30% reduction in wall clock time, that's pretty neat (and I'm still not running this at full power but 450 W is a bit much for that puny connector).

I have two of modded RTX 4090 with 48GB VRAM. But I'm using default drivers. I'm using models, which uses almost full 48GB.

Now i'm curious - am I missing some settings? Do I need to check any of these BAR things?

You're probably not using peer-to-peer then. Try the p2pbandwidthtest that comes with the NV development package to see if you are or not.

If they are forked from my repo whatever driver version they pin doesn't matter, the pin is just clanker stuff really. What it's used for is just extracting the right booter firmware and loading it, but the whole booter part is reset before handing over to nvidia driver so the versions dont have to match at all.

Apparently they were, it compiles and installs but I don't see any improvement just yet. I'll keep plugging away at it.

https://github.com/JustAResearcher/gpu-unlock-ada

That's a fork of GP's work.

But it contains the 4090 variant.

What about the 5090?

Hopper and blackwell have a very different security chain so those are out of scope for this approach at least.

But I would suspect China has figured something out given the reports of them buying 5090s in droves and seemingly modifying them.

I've looked a lot at that and it seems to be mostly related to packaging rather than performance, as far as I know there are no 'improved' 5090's for sale in any kind of volume, and the ones that they made that I know of (a 64 and a 96 G version) never made it to production for some reason. I'd love for that to happen though.

Maybe they got hold of the vbios for those higher vram versions similar to the 48gb 4090s?

There is a lot of speculation about this but nobody seems to really have hard insight. As long as the hardware supports it it should in theory be possible to unlock it but like with those 48G 4090's that can lead to weird side effects. For instance, you would never know that these cards are modded until you start using p2p and then suddenly it all falls apart. The reason is that for p2p to work they map the memory space of one card into the memory space of another. But if the BAR1 is < than the total VRAM that trick will work only until you run past the TOP of that block and then you'll have a crash.

I suspect that modding a 5090 in such a way that you can sell it is a lot harder than doing the same with a 4090.

48gb 4090 uses stock vbios

Is this relevant for 4090? It has FP8 which is already double the rate of FP16. Can this hack double the rate for FP8 too?

Answering my own question, FP8 with BF16 accumulate is double rate already, but FP8 with FP32 accumulate is half rate, and this hack doubles it to match BF16 accumulate.

https://github.com/JustAResearcher/gpu-unlock-ada

I have a 3090 founders edition sitting on a shelf. The FE cards had VRM cooling issues, IIRC. Would this unlock have any uplift even if I undervolt and don't increase the power draw limits much? The use-case I'm thinking is to run a model to power a local only voice assistant for my family. I'm not really familiar with what instructions are used most heavily by LLMs, but is my read correct that this would be expected to help with TTFT? I'm curious if there's anything about this that helps in particular with audio processing models.

And maybe I'm overthinking the cooling issue. It's not like I'm hammering it 24/7. Probably fine to burst high on power draw on use.

Btw, I always love moments like this where library or project owners stumble on their work and chime in on the comments. I really appreciate your work here. It's given me motivation to actually do something with parts that have been sitting on a shelf.

You can trivially fix most of the VRM cooling issues btw by pulling the heat sink off, cleaning thoroughly and re applying fresh thermal pads. At this point almost all of 3090s are degraded in some way since those pads have a lifespan. The good news is it has nothing to do with the GPU itself.

I know it's probably not that difficult, but I always feel anxious doing things like that with expensive components. The thought of being slightly misaligned and causing issues gives me pause. I guess now that it's out of warranty, I have a bit less to lose.

275W is the sweet spot on all my 3090s (not 3090 TIs).

I need to repaste two at 275W because they have some thermal issues, not main temperature sensor, either. Likely memory junction.

They are running completely fine at 250W with 13% decrease in compute output. Given the decrease of -25W... that's about a 3% net loss for exaFLOPs/kWh.

Any idea if something like this is happening on GB10 (DGX Spark) and needs similar treatment?

edit: seems you have provided some clarity that even if, unlikely soon unless China has and does the world a favor

DGX Spark is going to be very hard.

Did you use AI tools to do it? The README reads like Claudish. It's amusing to see Nvidia-powered AI used to unlock their GPUs.

really cool! would love to talk more if possible, sent a donation through ur github but if u can send me a dm on @chrs3775 on discord that would be appreciated

Reply on news.ycombinator.com