A Nvidia Engineer Wrote a Vulkan Driver That Works on Older Raspberry Pi | Hacker News Reader

A Nvidia Engineer Wrote a Vulkan Driver That Works on Older Raspberry Pi

phoronix.com

385 pointsreddotX104 comments

A few years back I wrote an N64 emulator graphics module for the Raspberry Pi because I was unhappy with the poor performance of the existing ones and I would have really wanted this. Drawcall overhead for Broadcom's official drivers is extreme: you can't have more than a single-digit number of drawcalls per frame and still maintain 60 FPS. I ended up having to go to ubershaders for everything just to maintain 30 FPS. I'm certain the hardware was capable of much more, but the drivers were holding it back.

This is one of RPi limitation - closed source firmware and drivers. But hardware is much better...

Only partially true on the Pi4: The complete OpenGL stack is now open source using Mesa (it was optional on previous models). Video decoding is slowly moving over from the closed MMAL stack to KMS and V4L2. The boot firmware is still closed though.

> The boot firmware is still closed though.

Not much worse than a PC then in that regard.

UPDATE - mine fault. CPU doesn't have AES instructions!

Actually it is.

For example some RPi CPUs have AES acceleration instructions built similar as AES-NI on x86. But due firmware limit we can't use them.

Probably it's licensing and pricing issue...

I looked it up. That's not true. The hardware isn't there, and would require the RPi foundation purchase a license to implement.

See the engineer reply in this thread:

https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=2078...

> That is licence for the HW, so you buy it then add the requisite HW to the die, so no, it cannot be added later without redeisgning the HW, which would cost about $1M Plus the licence....

Can't edit my own post, but please stop downvoting mobilio. They admitted they were wrong, and there's no shame in that.

Even if they had been wrong, they shouldn't necessarily have been downvoted, because it still ended up adding to the discussion.

It's an export law issue. Raspberry Pi is an educational product that should be exportable all around the world. They can't let crypto regulation laws upend that goal.

There is open RPi boot firmware here:

https://github.com/librerpi/rpi-open-firmware/

Wasn't one of the issues with earlier Pi versions that the driver just serialized high-level GLES-like commands to a front-end processor that ran a huge closed firmware blob? All GPUs have built-in command processors but this was closer to a full driver running as a firmware blob on an auxiliary CPU.

Video decoding is still a binary blob. There is no documentation to start writing your own decoders.

The video core documentation for the gpu is open and you can actually write code for the GPU (and an open source assembler exists)

Do you mean hardware module? Have you got any blogs or docs about this? Came across the same thing doing a baremetal kernel (fun!) and this never crossed my mind!

> I'm certain the hardware was capable of much more, but the drivers were holding it back.

Exactly like iPhone story on performance decreasing "for battery life" for older devices, each time on next device model rolled out.

Or like samsung phones, where they just stop giving you updates after 2 major releases

Excellent effort.

I wonder whether there will compositor support this driver, to take advantage in the desktop environment. I had made several attempts to get a smoother desktop experience[1] on RPi 3 -

•LXDE + Openbox on Raspbian + X server

•Xfce4 + VC4 + X server + Arch Linux ARM + USB SSD

•Enlightenment + Wayland + Arch Linux ARM + USB SSD

Although Elightenment on Wayland with OpenGL was the smoothest of them all, it's not usable(frequent crashes with RPi) and since the frame buffer was limited to 2048x2048 none of them supported my 2560x1080 monitor.

Xfce4 + VC4 on Arch Linux is more usable, but is still not as stable as default Raspbian. I didn't see any productivity merits in continuing this adventure and decided to reclaim the memory from GPU to revert into headless[Arch+SSD] for a motion eye setup processing 3 720p camera streams simultaneously with average of ~ 50% CPU on all 4 cores when not watching the feed live(but motion active).

[1]https://abishekmuthian.com/getting-smoother-desktop-experien...

I found arch to be quite stable on mine, though I didn't use a desktop most of the time. When I did, i3 was by far the fastest, so maybe give that a try.

I agree reg Arch ARM, may it's just VC4 that's causing the issue in desktop environment. I'll give i3 a try if I pursue this, but IMHO RPi < Pi4 are best suited for headless operations.

Very nice, this can speed up some of the old Pis still hanging around quite significantly if software authors start making use of Vulkan on ARM.

I wonder how Nvidia is looking at this with their terrible anti-open source mindset. I hooe an engineer of theirs with experience from their company writing a video driver doesn't get the author any repercussions.

Not just old - this also supports Zero and Zero W, which are in a category of their own.

repo: https://github.com/Yours3lf/rpi-vk-driver

Just makes me wonder whether there could be any conflict of interest here and who owns the copyright. I know some of the big corps assume ownership of whatever their employees produce even outside working hours. Or Maybe the project was signed off ?

How hard would it be to reverse engineer CUDA and make something like WINE that translates all CUDA operations into OpenCL or directly into third-party GPU instructions?

Obviously wouldn't be as fast as on NVIDIA hardware but it would potentially be much faster than the CPU versions of those pieces of software.

It's easier to replace the CUDA kernels with probably designed OpenCL kernels instead. Edit: For third party: Thats what the driver does, compiles OpenGL and OpenCL code into GPU machine code. In case of mesa based on reverse engineering.

> It's easier to replace the CUDA kernels

If it's your software, yes. But not if you're trying to fix an already-written framework or run other peoples' machine learning models.

Why reverse-engineering something that is very well documented? (and hell, PTX even has GCC and LLVM backends while we are at it)

The issue here is manpower, and not obfuscation in any way.

It’s a lot of work.

That’s actually it with the CUDA stuff. Nvidia made a massive investment, and is still doing so, so the work got done. OpenCL just doesn’t have the same cash backing it up.

Oh also this is basically what AMDs HIP does. The issue is that the project hasn’t been moving very fast.

Prepare for loads of reverse-engineering, as there are binary-only CUDA kernels in applications. This does tend to break when new GPUs come out, but a software update takes care of that. I assume some do that for obfuscation, while others are certainly using it for performance, see e.g. https://github.com/bryancatanzaro/nervana-lib-gpu-performanc... for some practical-ish examples.

Any chances to see something similar ported to older Mali GPUs such as the ones contained in lower end Allwinner ARM SOCs? So far it seems only newer ones are (officially) supported. https://developer.arm.com/solutions/graphics-and-gaming/apis...

Does it mean you’ll be able to speed up the compositor of gui and ff for example?

It doesn't look like this supports GLSL, so that would have to be addressed first. Furthermore, there's no Vulkan backend yet for WebRender.

Actually it might be, Zink just got OpenGL 3.0 support.

Zink is an OpenGL Implementation on top of Vulkan and that might actually be a good way to do full OpenGL on these devices.

But it's only a small subset of Vulkan that this driver implements. I would be surprised if it's enough for an OpenGL implementation.

Zink has next to no optimization effort so far and is thus very slow from what I've read. Maybe in a few years, since they currently seem to be focusing on compatibility.

Which is the right way to go about it, especially when Vulkan is still in the process of being adopted. Make it right; then make it fast.

Glad to see any green/red team engineers doing any *nix stuff frankly :)

Seems super petty given the wholesomeness but would have preferred if he invested the energy into RPi4 to be honest though given how accessible it is now.

The Linux team at Nvidia is actually full of awesome people, used to know some of them. They’re just often hamstrung by business objectives.

Perhaps it will come to RPi4 as well?

RPi4 has a different GPU than the older models. Someone else is already working on an official Vulkan driver for it.

I wonder whether this situation of reverse-enginnering GPU drivers is sustainable. GPUs on ARM SOCs tend to be quite different from each other and most vendors have no interest in collaborating with the linux community. On X86 at least we have Intel and AMD willing to invest in linux drivers.

IMHO it is just unethical to have to reverse engineer your hardware to program for it.

NVidia ships heaps of Linux stuff, mostly via their machine learning teams. For example they have their NVidia Jetson devices which all run Linux: https://www.nvidia.com/en-au/autonomous-machines/embedded-sy...

I've always disagreed with the grammar rule in this situation. "An Nvidia engineer" just sounds and flows better than "A Nvidia engineer"

I'm pretty sure "an NVidia" is grammatical here.

What is the grammar rule?

I've always assumed that you use "an" whenever a noun starts with a vowel sound, which "Nvidia" does ("n" being pronounced as "en"), thus making "an Nvidia" the grammatically correct one.

This is actually a deeper issue about the confusion between the language itself (english phonology requires what you're describing) and the written form. No english speaker in the world, unexposed to the written form of "Nvidia" would ever say "A Nvidia."

The rule is just "which one is easier to say? use that one". "A nvidia" is just wrong.

That entirely depends on how you pronounce "nvidia" in your head. I can imagine someone who has never heard the name spoken out loud think of it as "ni-vidia" (I have done worse) and "a Nividia" is easier than "an Nividia"

I mean, that's just pronouncing it wrong.

You are right: my point is that the easy way to say something can very well be the wrong way to say it.

I meant is it easier to say "a nvidia" or "an Nvidia", assuming you are pronouncing nVidia correctly.

The ppronunciation is "en-vidia", so yes, "an Nvidia" would be correct.

If the company was pronounced ni-vidia (or any other vowel in place of that first `i` vowel, then "a Nvidia" would've been correct.

Just like it's "a user" not "an".

or 'an uber' instead of 'a uber.'

Thought it is a Uber as you say yu-ber in English. It sounds different if in German?

You absolutely do not say you-ber in english

I think I learned something new here. I've always thought the rule was "use `an` if it _is_ a vowel, else use `a`".

If I understand this correctly now, it's "use `an` if it makes a vowel _sound_, else use `a`".

It’s definitely about vowel sounds, because we don’t say “a hour” or “a herb”.

deleted; i misread the parent comment

"hour" starts with an 'o'-sound, since the 'h' is (mostly) silent, so "an hour" is correct.

> a herb

I do, but that's because I pronounce the 'h' in 'herb'.

> It’s definitely about vowel sounds

Exactly, and Nvidia begins with an e sound.

And we say "a union" because it begins with a y sound and not because it begins with the letter u.

In the UK, people do say "a herb", but that's because we pronounce the h: https://en.wiktionary.org/wiki/herb#Pronunciation

An historic day!

That's just weird over-correction. "A historic day" is correct but for some reason people heard "An historic day" and the idea spread that it was a special exception.

Kind of like how you see lots of people on Reddit saying "water isn't wet, it just makes other things wet". They heard someone else say it and it sounds smart so they repeat it, without spending 10 seconds to look it up in a dictionary.

To me it may be a tell for the writers at one point pronouncing it as "'istoric", similar to those who say human as "'uman", and then as you said likely fanning out as an over-correction, possibly because that dialect was associated with prestige.

Yes maybe once upon a time but it doesn't explain why people say "an historic" today.

I an struggling to understand what is wrong with the phrase "water isn't wet, it just makes other things wet". Can you explain?

Water is wet. There's nothing wrong with it grammatically. I just meant it was an example of people blindly copying untrue but smart-sounding facts.

"we" being Americans: the rest of the English-speaking world says "a herb" as the "h" in "herb" is not silent

Yeah the rule is just "which one sounds better?". That's why "an" exists in the first place - because saying "a nvidia" is super awkward.

HN battling over grammar which is a standard. Focus on topic!

Be careful though. There are words that start with a vowel when written, but are pronounced with a consonant. Eg "a universe".

"An Nvidia engineer" would actually be grammatically correct here, since the pronunciation of "Nvidia" starts with a vowel sound and not a consonant one. It's the pronunciation of the word that matters for a/an, not the spelling.

In case anybody was wondering, the official pronunciation of the word "Nvidia" is apparently "en-VID-eeyah" [1].

[1] http://international.download.nvidia.com/partnerforce-us/Bra...

I didn't know that, I always got told it depends on the first letter (and it didn't make sense for me for some words).. (Obviously not a native speaker)

Don't worry too much: I am a native speaker and I was taught the same in elementary school!

Nope, it's based on the first sound. (Hence "an hour" rather than "a hour").

Don't tell the knuckle draggers still pretending that "history" has a silent-H.

That's an accent thing. In some places the "h" is silent, and in other places it's not.

Is there any English dialect in which it's silent in "a history"?

This is also why it's "a university".

Wait, what? So the rule isn't that the pronunciation starts with a vowel sound? Instead it's a more murky thing about how easy it rolls off the tongue?

University starts with the 'y' sound, hence "a university." On the other hand, "an unimpressive fact" uses "an" because "unimpressive" starts with the 'u' (vowel) sound.

Doesn't "university" start with the "y" consonant sound?

University has a kind of "y" sound, like the word "yes", which isn't considered a vowel sound.

That’s why the pronunciation of “an history” is such a contentious issue in British political linguistics to this day. Crucially, as has been observed correctly, the rule is that one should use the appropriate indefinite article. That in turn depends on whether one pronounces, as pirates, or has a silent ‘h’, revealing one’s origins even once the guise of Received Pronunciation had been defensively adopted.

...and the name comes from https://en.wikipedia.org/wiki/Invidia (pronounced almost identically), also the eye logo.

(I find it amusing that the last item on that logo guidelines is basically saying no to its own site.)

I wish my English teacher told me this 20 years ago. It all makes sense now. Mind blown.

I certainly would have preferred “an engineer that currently works with Nvidia”

The way it is written currently makes it sound as though Nvidia owns this person.

> The way it is written currently makes it sound as though Nvidia owns this person.

Not to anyone with a shred of common sense. That's a pretty common way of putting it that's far less clunky than the way you wrote it.

> The way it is written currently makes it sound as though Nvidia owns this person.

You can't be serious.

And now they own his home office as well!

Titles with character length limitations do create tension that pulls sentence construction toward concise information density over strict readability, but avoiding unintended implicit statements in titles is a concern of mine.

This usage is acceptable because if you simply delete the word nvidia, we lose valuable context which tells us why we should pay attention to this post over any other engineer’s post with analogous drivers for this use case: the pedigree of the engineer suggests that the driver will be performant and of a similar quality as those other drivers produced by the engineer’s company.

The English word “apron” comes from the French “napron.” “A napron” became “an apron.”

The French word is napperon.

Isnt "an orange" also supposed to have come this way?

There is a Canadian joke that is related to this very thing:

    Where did the Québécois get that H sound for Ottawa?
    They took it out of here.

When speaking with a French Canadian accent, Ottawa sounds like Hottawa, and Here sounds like Ear. There is a similar more adult-oriented joke about the word Happiness. It is utterly baffling to Anglophones, especially since whatever rule they are following doesn't apply to the word Ontario, which pretty much sounds the same as in English.

If you've never heard Quebec French, imagine Celine Dion telling the joke and you'll get it.

Maybe it's based on emphasis? 'Ott.a.wa vs on'ta.ri.o

The Happiness joke is based both on the missing H and the different emphasis on syllables, so you are probably right or at least close. The subtraction of H in words like Here and Happiness seems like a different rule than its addition on hard vs soft initial-O words, though.

What Happiness joke? I’ve lived in Montreal for 15 years and never heard it.

It sounds like "a penis" to Anglos unaccustomed to hearing the French language rhythm, plus the unpronounced H.

That IS the right grammar. A Nvidia is wrong.

An?

for fuck's sake, no one has to follow standards!

Great effort, but https://github.com/Yours3lf/rpi-vk-driver/tree/master/extern... makes me side

Just use NixOS everyone, and never vendor anything ever again no matter how many packages you need to fork.

Reply on news.ycombinator.com