I find it odd that given the billions of dollars involved, no competitor has managed to replicate the functions of CUDA.
Is it that hard to do, or is the software lock-in so great?
Is it that hard to do, or is the software lock-in so great?
The non-nvidia hardware vendors really don't want cuda to win. AMD went for open source + collaborative in a big way, opencl then hsa. Both broadly ignored. I'm not sure what Intel are playing at with spirv - that stack doesn't make any sense to me whatsoever.
Cuda is alright though, in a kind of crufty obfuscation over SSA sense. Way less annoying than opencl certainly. You can run it on amdgpu hardware if you want to - https://docs.scale-lang.com/stable/ and https://github.com/vosen/ZLUDA already exist. I'm hacking on scale these days.
But the concrete qualities of CUDA and Nvidia's offerings generally is a move toward general purpose parallel computing. Parallel processing is "the future" and approach of just do loop and have each iteration be parallel is dead simple.
Which is to say Nvidia has invested a lot in making "easy things easy along with hard things no harder".
In contrast, other chip makers seem to be acculturated to the natural lock-in of having a dumb, convoluted interface to compensate for a given chip being high performance.
Notably, AMD funded a CUDA clone, ZLUDA, and then quashed it[1]. Comments at the time here involved a lot of "they would always be playing catch up".
I think the mentality of chip makers generally is that they'd rather control a small slice of a market than fight competitively for a large slice. It makes sense in that they invest years in advance and expect those investments to pay high profits.
[1] https://www.tomshardware.com/pc-components/gpus/amd-asks-dev...
> it's 20 years worth of institutional knowledge with a stable external api
> There are very few companies outside of 00s Microsoft who have managed to support 20 years worth of backward compatibility along with the bleeding edge.
To me that sounds like massive investment
They overlook CUDA is a polyglot ecosystem composed by C, C++ and Fortran as main languages, with Python JIT DSL since this year, compiler infrastructure for any compiler backend that wishes to target it of which there are a few including strange stuff like Haskell, IDE integration with Eclipse and Visual Studio, graphical debugging just like on the CPU.
It is like when Khronos puts out those spaghetti riddled standards, expecting each vendor/open source community, to create some kind of SDK, versus the vertical integration of console devkits and proprietary APIs, and then asking why professional studios have no qualms with proprietary tooling.
But you're right there was already something in place.
I mean efforts like rust-gpu: https://github.com/Rust-GPU/rust-gpu/
Combine such language with Vulkan (using Rust as well) and why would you need CUDA?
Probably just needs a couple short decades of refinement…
Hence they bought PGI, and improved their compiler.
Intel eventually did the same with Open API (which isn't plain OpenCL, rather an extension with Intel goodies).
I was on a Khronos webminar where the panel showed disbelief why anyone would care about Fortran, oh well.
Which in the end was worthless because both Intel and AMD botched all OpenCL 2.x efforts.
Hence why OpenCL 3.0 is basically OpenCL 1.0 rebranded, and SYSCL went its own way.
It took a commercial company, Codeaplay a former compiler vendor for games consoles, to actually come up with a good tooling for SYSCL.
Which Intel in the middle of extending SYSCL with their Data Paralell C++, eventually acquired.
Those products are in the foundation of One API, and naturally go beyond what barebones OpenCL happens to be.
The mismanagement Khronos has done with OpenCL is one of the reasons Apple lost ties with Khronos.
Besides, I'd say Rust is a nicer language than CUDA dialects.
Khronos standards, CUDA, ROCm, One API, Metal, none of them has Rust on their sights.
World did not back OpenCL, because it was stuck on a primitive C99 text based tooling, without an ecosystem.
Also Google decided to push their Renderscript C99 dialect instead, while Intel and AMD were busy delivering janky tools and broken drivers.
Also SPIR worked so great for OpenCL 2.x, that Khronos rebooted the whole mess back to OpenCL 1.x with OpenCL 3.0 rebranding.
https://www.khronos.org/events/shading-languages-symposium-2...
Cross compiling Rust into PTX is not enough to make researchers leave CUDA.
Being language agnostic is also not the task of the language, but task of IR. There is already a bunch of languages, such as Slang. The point is to use Rust itself for it.
Slang belongs to NVidia, and was nicely given to Khronos, because almost everyone started relying on HLSL, given that Khronos decided not to spend any additional resources on GLSL.
Just like with Mantle and Vulkan, Khronos seems that without external help they aren't able to produce anything meaningful since Long Peak days.
The language is general, but the current focus is really on programming GPUs.
The reason why people continue to use CUDA and Pytorch and so on is because they are literally too stupid and too lazy to do it any other way
When you're done, you can create IDE plugins, and a graphical debugger with feature parity to NInsights.
The argument you are making sounds to me like, "well good luck making a Vulkan application without cmake, ninja, meson, git, visual studio, clion" etc, when in reality a 5 line bash script to gcc works just fine
Lacking understanding is doomed to failure.
Nvidia's own people are the ones who have made Vulkan performance so close to CUDA's. AMD is behind, but the data shows that they're off in performance proportional to the cost of the device. If they implement coop mat 2, then they would bridge the gap.
99.9% of people who use Pytorch and so on could achieve good enough performance using a "simple vulkan backend" for whatever Python stuff they're used to writing. That would strip out millions of lines of code.
The reason nobody has done this outside of a few github projects that Nvidia themselves have contributed to, is because there isn't a whole lot of money in iterative performance gains, when in reality better algorithmic approaches are being invented quite near every month or so.