I'd be curious to see what kinds of implementations someone could come up with in CUDA using templates, though. It could easily blow the Julia performance out of the water.
And this is tuned towards package development, specifically generic package development. This is quite a hot topic in scientific computing. The challenge isn't for people who use packages, it's for people who develop packages and are looking to make them generally useful (i.e. not just single project).
Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
Note that I absolutely love Julia because I find it far easier to write than a language like C++, but idiomatic Julia code still has low overhead, unlike, say, Python, which introduces orders of magnitude of overhead compared to low level languages, and you have to move away from simple idiomatic code to achieve reasonable performance. However, I find it unreasonable to focus on SIMD performance - the performance gain over non-vectorized code isn't that big, and if you do want to eek out that much extra performance, you should focus on using the GPU.
That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing.
>Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU.
Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-...
>but idiomatic Julia code still has low overhead
Where?
And those situations tend to get a very minimal speedup from SIMD.
>Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-....
Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance.
>Where?
I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).