Hacker News Reader

Comment by kruxigt | Hacker News Reader

For me it's really interesting to read all the complaints about precompilation times etc. I'm definitely not saying the complaints are false, just that there are also some of us out there where they matter close to 0 (at current level). I use Julia from the REPL, running mostly self constructed algorithms. Only fairly basic modules are used.

My main project is a backtesting framework for algorithmic stock trading. A typical run took more than a month of runtime on my computer when the code was in optimized Numpy and now takes less than 1 minute in optimized (but likely far from optimal) Julia code. The only way to come at all close to the speed of Julia with Python would probably be something like Numba, which I figured would be more immature than Julia. Python is only usable for heavy computing on large data when the glue can be in the outermost loop so to speak.

View on HN

I find the month to minute transformation rather shocking. Are you sure you weren't doing something algorithmicly wrong in python? Did you profile it? Like I would never leave something running for a month without putting a fair bit of time into making sure it was fast.

I don't think I did anything algorithmically wrong. If I could have done it with more of the data handling being done in c functions and just using Python as outer glue? Most likely but then that would not make the framework very extensible to other models.

Not only did I profile it, put a lot of time into optimization and learned how to use cloud computing, but I also learned a new language (Julia) and ported the whole application. :)

In Julia I can use things as data oriented design (few cache misses) to further reduce runtime. I'm not sure you get much out of such efforts in Python.

This. Having tight control over memory layout + heap allocations + vector instructions can literally decrease runtume by several orders of magnitude. Performance is so much more than having the "right algorithm". The low-level details matter so much.

Time series analysis especially goes well with Julia, as the number of dependencies is in the same order of magnitude as the number of data points.

Numpy / torch shines when the slowest part of the data processing is big matrix multiplication. In that case actually waiting for Julia compilation can slow down experimental iterations.

That said, the Julia community has done some pretty amazing work in that subfield, last I checked they were getting performance in pure-Julia very close to the of BLAS/LAPACK, which is pretty incredible.

Julia is improving, but I wouldn't compare it to BLAS/LAPACK, cuBLAS is a much better comparision.

PyTorch uses tensor cores, supports mixed precision multiplication and bfloat16 representation.

Julia's implementation in CUDA.jl doesn't even use shared memory, which is important, as using that 10MB cache can reduce the needed main memory bandwidth significantly.

Also the new 30xx NVIDIA series supports sparse matrix multiplication, which is great if you have ReLU operations in your data processing for example. I'm not sure if PyTorch has support for that, but I'm sure it will be added, just like how mixed precision multiplication was added this year.

We're talking about an order of magnitude difference if you add all these small things together that PyTorch / cuBLAS supports. Of course Julia can call into these libraries, but then all the nice optimizations that make Julia Julia go away.

I had to think really hard of these things actually, because I love the Julia language, but the more I see ML pipelines beating the previous state-of-the-art models by 5x, the more I understand why an interpreted language with a great linear algebra library won the data processing race (I wish it would have been Ruby though).

No, the parent comment is correct. Tulllio.jl has been getting up to openBLAS levels of performance

Julia also is on the cusp of support for tensor cores, mixed precision multiplication and bfloat16 codegen. New PR just merged last week

https://discourse.julialang.org/t/blas-vs-cublas-benchmark/4...

Here's an OpenBLAS vs CuBLAS performance comparision. Basically CPU is at this point outdated technology for matrix multiplication. Your second comment is more interesting, I'd be happy if Julia gets to the point when it can beat PyTorch or even JAX on training Lambda networks or Performers.

I understand CPU is slower, but it's not going anywhere, and matmul is just a benchmark.

It's extremely impressive that a high level (using index notation for multiple backends) Julia library can compete with hand tuned kernels. Also bodes well for Julia compiler tech generally that can easily be extended

Julia is going to get to the point of beating pytorch, just taking a but more time due to smaller team and approaching it from a more general position..so when it does it will be more flexible and ergonomic and easily extensible to new techniques, in pure Julia.

1.6 will be a big step as much of the compiler hacks underlying the current ad/gpu codegen (which just fell out accidentally of lispy design) will be replaced with proper tooling for composable compiler passes on typed IR. This will be a phase change imo.

There's already a new faster AD that's almost ready for debut based on 1.6 tech

That sounds amazing, thanks for the update!

I stopped using Julia because it was very frustrating that I payed thousands of dollars for a laptop with GeForce RTX 2070 card and the I couldn't make use of it, it felt like I wasted a lot of money...that was my main decision for moving to PyTorch. I know that Julia as a language should be able to beat it easily, but it felt to me that the community was prioritizing CPU over GPU for a long time. I'm happy that it's changing now.

The GPU stuff is basically just a teaser into the future, it's still much more immature than that CPU stuff for Tullio.

> Basically CPU is at this point outdated technology for matrix multiplication.

This is an incredibly naive statement. There are many many circumstances where you need fast matmul on the CPU even if you have a GPU available because it takes too long to send the memory to the GPU and fetch it once the kernel runs.

If you have many chained matmuls, then yes the GPU is your friend.

That said, regardless of how useful the CPU is in practice for matrix multiplication, BLAS kernels are some of the most overengineered, most optimized pieces of code in wide use.

Tullio is incredibly flexibly and general. The fact that matrix multiplication falls out as a special case in Tullio and can outperform OpenBLAS and match MKL is absolutely stunning.

Can you give concrete examples when data processing on the CPU is clearly better than on the GPU?

Before I was working with time series, and there the CPU really shined, but right now I'm classifying sick patients using DNA methylation data (about 100-300 GB / dataset). I'm using a convolutional network that already gives me state of the art results, but I want to experiment with self attention based neural networks. While I'm doing this, most of the biologists are still using logistic regression that can be computed on the CPU.

I see more and more domains (even simulations) where neural networks can outperform classical methods if you know how to apply them, and CPUs can never match the GPU performance. Also as I wrote, the slowest part of all these networks is the matrix multiplication (self attention is especially depending on it).

> Can you give concrete examples when data processing on the CPU is clearly better than on the GPU?

Pretty much anything where the matmul is important to performance, but doesn't dominate it and requires serial steps. A classic example off the top of my head would be solving a matrix differential equation[1].

Here's an example in Julia using DifferentialEquations.jl and CUDA.jl for the GPU part:

This is solving the differential equation du/dt = A * u for the cases where u and A are arrays on the GPU versus when they are arrays on the CPU. I'm doing the solving in-place to try and minimize the amount of expensive allocations.

    using DifferentialEquations, CUDA

    function mysolve(u0, A, tspan)
        f!(du, u, A, t) = mul!(du, A, u)
        prob = ODEProblem(f!, u0, tspan, A)
        solve(prob)
    end
    mysolve (generic function with 1 method)

    julia> let n = 50, u0 = rand(Float32, n), A = randn(Float32, n, n)
        # Make GPU versions of u0 and A
        cuu0 = cu(u0)
        cuA = cu(A)

        # Do a run of the code so there's no compiler latency biasing the results
        mysolve(  u0,   A, (0.0, 1.0))
        mysolve(cuu0, cuA, (0.0, 1.0))

        # Now run the compiler code and time it's execution:
        @time mysolve(  u0,   A, (0.0, 1.0))
        @time mysolve(cuu0, cuA, (0.0, 1.0))
    end;
    0.000314 seconds (621 allocations: 98.281 KiB)
    0.007218 seconds (10.33 k allocations: 382.266 KiB)

In this case, the CPU code was ~20x faster than the GPU code. True, I could get a better GPU (I'm using a RTX 2060), but I could also get a better CPU (Ryzen 5 2600).

There's not much point in using a GPU for this example, and I think that just comes down to the fact that there's a non-trivial amount of serial work that needs to be done between the matmuls.

[1] https://en.wikipedia.org/wiki/Matrix_differential_equation

I wrote big (10000x10000x10000) matrix multiplication and data processing (gigabytes of data) in the original comment, in which case the matrix size is small compared to the number of multiplications that the device has to make. Try n=10000 in your example (although I'm not sure about the differential solver part, as I don't run simulations), it's more interesting. Also differential expressions are more useful for simulation than data processing in my experience.

Of course CPU is faster for small matrices, but again, if I have hundreds of gigabytes of data that you want to process, CPU is always slower.

https://discourse.julialang.org/t/realistically-how-close-is...

Which implementation of BLAS/LAPACK? Because if it's the ref. implementation then it's not that impressive, the performance of that is very poor.

OpenBLAS and MKL.

Good point! Stuff that can be efficiently expressed as matrix operations is trivial for "Python" (eg BLAS) to handle. But I don't know of a good way to express a rolling maxima of a time series that way. And even less so if it is a streaming data feed.

What kind of architecture are you using for your backtesting framework? I have been collecting data to attempt to build something similar to that with close to around 2.3TB of tick data stretching around 20 years across 75 major pairs (indices, fx, major commods, etc) and I am always overwhelmed about design of systems to process large datasets like this.

I realize it's a bit off topic, but I just have to ask - where did you get so much ticker data?

It's been a work in progress for quite a while. Involved writing many different scrapers and other data ingestion systems that systematically downloaded everything I could get my hands on. I'm building an API to eventually sell some of the information, so I'd prefer not to say specific sources.

I'm not sure what you mean with architecture? The computers I currently run on are ordinary Macs of the latest generations. (Before Julia came and saved me I ran on Google cloud)

That is way more data than I am currently using! I'm currently a hobo using data from Yahoo finance. :) That sounds interesting! Could you contact me at kruxigt at gmail.com and tell me more?

I'd love to. I'll contact you soon.

Reply on news.ycombinator.com