Hacker News Reader

Comment by icapybara | Hacker News Reader

Does “90,000x speed up over pure python” with matrix multiplication mean vs using numpy arrays in python?

No, plain python code: https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...

I'd like to know how fast numpy is here, but they didn't compare... which is weird because that's what almost everyone would use.

It’s not just weird, it’s an intentionally dishonest evaluation to build hype around their language.

If you ask your average Python programmer what “pure Python” means they’d think numpy is included.

Their Mojo code just does the same optimizations numpy certainly has.

My experience is that most Python programmers would consider "pure Python" to basically be using the standard library.

There's also the issue of doing complicated work with NumPy and you start looping and revert back to slow (pure) Python because you are crossing the NumPy/ Python interface. Tools like Cython, Numba, and probably Mojo help to solve this.

For scientific users like they're targeting, `numpy` effectively _is_ part of the standard library.

The python/numpy boundary is a problem, which is sort of the value proposition of efforts like mojo or julia - no argument.

But nobody doing any significant numeric work would ever consider a "pure" python matrix multiply meaningful. It's disingenuous to present that as your comparison, at least without also including the way it's actually done in practice.

Honestly, it undermines their presentation with anyone involved in the area.

Pure Python means written wholly in Python to most Python programmers in my experience. For some Python programmers this excludes the 3rd party libraries they care about.

AI developers expect these sorts of comparisons to be taken against accelerated code anyways. Modern Python NN frameworks don't spend significant time in the Python interpreter, so that's not where users would expect the bottleneck to be for their comparison.

It's like claiming that your code is faster than MATLAB's for loops. Why write gemm yourself?

Even if this language is a good idea, I worry that they don't seem to understand the audience.

The difference and the whole sales pitch is based on that detail - that you write everything in one language; there is no high level and low level language switch, everything is written in one langauge.

> It’s not just weird, it’s an intentionally dishonest evaluation to build hype around their language.

Not too dissimilar to Swift, which was launched as being faster than C/C++.

But that is basically the core value proposition of mojo. You don’t need to drop down into another language to do low level, high performance kernels, you can do it all in mojo. Scalable in terms of target domains, (somewhat similar in that regard to swift).

I skimmed the linked code. Is there enough info there to run the same benchmark with numpy? And then if we find out that numpy is also 90000X faster then it's fair to conclude that numpy and Mojo have the same performance on this task? I'm a benchmarking n00b, go easy on me. Just trying to see if my basic understanding is correct...

I think it is just pure python

I don't know about mojo, but they seem to import python module for this benchmark. https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...

And for some reason they don't compare against NumPy https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...

What would be more interesting is to use NumPy and do the vectorized opreations.

I did just that, it's 120x faster than numpy. See my comment to OP.

I really don't understand why they do this. I was really excited for the compilation and type checking features but this whole speedup thing is pretty dumb. And I know the people developing mojo knows that too. Kind of seems like some marketing team is pushing them to do all this but their core customers are people who knows what python and numpy is and this kind of talk puts just feels weird

I think that mojo in it's current form is not on par with numpy performance (if it was they would be saying that). Even if the performance is same I would still give it a try. Their whole marketing though is making me reconsider

They do this because they want to showcase writing code in single language that can do things that other languages can't. Other langauges are either low level or slow, but they are high level and fast.

Naive question, but how do they distinguish themselves from Julia, which is also in that space?

Primarily by compatibility with Python, which is the de facto standard in deep learning.

I think there's a Julia forum thread where they are casually benchmarking comparisons between the two. Not sure if it is still active though.

There is large overlap with julia, yes. Both are addressing two-language problem.

Mojo is python syntax first where they want to be proper superset, which gives them access to wide python ecosystem and community. If executed well, this alone can absorb community similarly to how ie. typescript absorbed javascript community. Also similar thing happened with objective-c -> swift - also led by Chris, which gives a lot of credibility to the whole initiative.

Julia is proper new language you need to learn, use new tooling around it, ecosystem is quite academia skewed, ie. writing web services is probably not the best idea etc.

Additionally Julia suffers from "time to first plot" problem, which alienates a lot of newcomers who are not familiar or simply don't want to switch to programming mode where it becomes less of a problem (repl/notebook style where runtime is always active).

Both are very interesting languages, but mojo's starting point and trajectory seem to be at different level, ie. adoption may be very sharp.

Julia can cache JIT code between executions nowadays.

Julia is a language that was developed by academics, with not the best aesthetics in the implementation. I see a lot of value in starting with a clean, well engineered layered approach. If anything this is something I would expect Lattner to be able to deliver, whereas Julia while certainly powerful is tremendously messy.

Python was also developed by academics.

PyTorch recently added support for JIT compiling numpy code[0]. And then there are libraries like Numba[1]. I wonder how Mojo compares with exiting OSS Python JIT libraries like these?

[0]: https://pytorch.org/blog/compiling-numpy-code/

[1]: https://numba.pydata.org/

No It's vs a pure python implementation[1]

[1] https://github.com/modularml/mojo/blob/5ce18c47a27c0c4123de1...

Seems a bit deceiving then, nobody does matrix multiplications in "pure python." They use numpy or some derivative of it.

Here's the same benchmark with numpy instead of native python (on M2 MBP)

    Python             4.216 GFLOPS
    Naive:             6.400 GFLOPS            1.52x faster than Python
    Vectorized:       22.232 GFLOPS            5.27x faster than Python
    Parallelized:     52.591 GFLOPS           12.47x faster than Python
    Tiled:            60.888 GFLOPS           14.44x faster than Python
    Unrolled:         62.514 GFLOPS           14.83x faster than Python
    Accumulated:     506.209 GFLOPS          120.07x faster than Python

Reply on news.ycombinator.com