Hacker News Reader

Comment by taylorius | Hacker News Reader

I read another commenter here mention that Python is 50x slower than Javascript. Now browser Javascript VMs underwent enormous speedup efforts over the last 12-15 years (Google chrome starting the race). Is there anything inherent to Python's design as a language (or any other technical reason) that prevents such a speedy VM being created for it as well?

View on HN

Yes, definitely. Python is much much more dynamic than Javascript -- the jump in dynamicness of Javascript to Python is about equivalent to Java to Javascript, and similarly requires a different set of techniques.

The single example I typically give is that in Python you can inspect stack frames after they've exited! This has concrete implications on your VM design.

Another example is that the multiple dispatch protocol for binary operations (such as addition) is sufficiently complicated that you generally cannot encode it into your compiler. We have been able to have a simplified version of that by building a tracing JIT for C, which is able to understand the dispatch protocol, but it is not always applicable.

Source: I've been working on Python optimization for several years as part of the Pyston project.

Unlike JavaScript however, Python is strongly typed.

Python's type system is not more strongly typed than JavaScript's in any way that makes it easier to optimise.

Yeah but in practice basically all JS code in the wild is strongly typed since the community has long since decided that relying on implicit type coercion is a massive footgun.

Also I think Python’s truthy/falsey semantics make it dangerously close to being weakly typed.

All that means is that you get more exceptions raised when you use ==. It doesn't help performance at all, it doesn't even really confer anything about types, only that __eq__ is implemented between fewer types.

Python exceptions are low-cost. It has to be that way because every for loop in Python throws an exception.

>The single example I typically give is that in Python you can inspect stack frames after they've exited! This has concrete implications on your VM design.

How about an implementation of python that doesn't let you do fancy things like that that are not needed in production?

The inspecting of stack frames is used by the logging library, and by most unit testing libraries. There are dozens of these small features which are challenging, and if you remove all of the challenging ones, you end up with something where pretty much no existing Python can run.

People have tried, pypy started with that goal before it became clear it wasn't practical.

I think logging inspects frames while they're still on the stack, not post-fact..?

Not more dynamic than Smalltalk or SELF, though.

Or dylan or julia

Please share the well understood criteria for "dynamic" in this context, that would let us order different programming languages as more dynamic or less dynamic.

Search the web for Ruby or Smalltalk or Objective-C “object model” or “message dispatch”

Please share your criteria for "dynamic", that would let us order different programming languages as more dynamic or less dynamic.

Is there any reason that Python's dynamicness (dynamicity?) is so much more of a hindrance than CL's? Is it just that CL is compiled?

The end of this post explains why Common Lisp can run much faster than Python.

https://markmail.org/message/dp56i26zhpf4fehb#query:+page:1+...

>> in Python you can inspect stack frames after they've exited!

How does that work and what is it used for?

Do you happen to know something about the graal python implementation? Based on other comments here, the often used C FFI can also hinder the work of Python implementations, and the graal project solves it by JIT compiling C code as well (from llvm ir)

From what I’ve seen last time I heard about graal python is that it is seldom faster than the pypy approach (AST-based vs tracing JIT), even though in case of TruffleRuby and TruffleJS it is insanely fast compared to the maturity of the projects.

Is your mentioned example of stack frame inspection make it map much worse to Java primitives (which happens behind the scenes of the truffle languages)?

I thought this presentation by Armin Ronacher (the creator of Flask and Jinja) was very enlightening:

How Python was Shaped by leaky Internals(2016):

https://www.youtube.com/watch?v=qCGofLIzX6g

See also this HN post (Attempts to make Python fast):

https://news.ycombinator.com/item?id=24848318

From a language design perspective I don't think there's much in the way of making up this ground. The hard part is not breaking any existing modules written in C/C++/Rust/etc that depend on the cpython ABI (memory layout and calling conventions of the runtime's implementation). There are jit compilers for python like pypy which are very fast (>5x they're attempting to gain here) but they often break things like numpy that have a lot of native modules.

Sounds pretty straightforward to get great speedups where no modules are used, and just make module calls slow (ie. you reconstruct the data structures the module will need on any entry to the module).

Then release a new 'fast' module API, and all the performance critical modules will soon move over.

It might sound easy but that would be a critical component of such a rewrite. Python's standard library is implemented using the same API so much of that would need to be rewritten. You then have the risk of introducing bugs, etc. This post is doing something similar. They're rewriting the internal implementation through progressive refactoring and will then look into more long term incompatible changes that can be wrapped in a translation layer. These projects are difficult as a lot of python code exists in the world and a minor change in behavior can have large unintended consequences due to Hyrum's law.

The big difference is that lots of python libraries use C under the hood, and a lot of internals of the language are leaked via the C api. It's a lot harder to do fancy things with a JIT when there are more people observing and depending on the internals.

Since everyone knows Python is slow, no care is taken to make things faster in frameworks etc., since one assumes the users are fine with the speed. If "they wanted it to be fast, they would use something else, right"?

So I stipulate that if the runtime becomes faster, those gains won't be visible to large swathes of code because python libraries do lots of weird things under the hood.

Yes. At the end of the following Usenet post, the developer of a Python implementation compares the language features of Common Lisp to those of Python in order to explain why Lisp code can be compiled to run much much faster than Python. The same kind of comparison can be done for JavaScript. The problem is a collection of Python features, each of which saps performance.

https://markmail.org/message/dp56i26zhpf4fehb#query:+page:1+...

Python is just too dynamic, and users do make use of it. There are JavaScript constructs that JITs give up upon, such as “with” - and this has steered JS programmers to avoid those constructs.

Unfortunately, there is no Python subset people can stick to as such. But you can use e.g. TransCrypt to compile Python to JavaScript; the docs tell you about an efficient subset. It should be possible to build an efficient Python JIT for such a subset (and you sort of have one already through TransCrypt and JS)

Smalltalk, remap all instances of a into b across the complete image, no matter where they are being used or which classes they actually are instances of.

    a become: b

It is a myth that only Python enjoys such dynamic capabilities, it happens to be a convenient excuse.

It's a lot more than an excuse; The simplest "a.b" has to check a lot of places before it gives an answer, and each of these needs its own in-line cache. Indeed, there have been experimental compilers for Python that gave much improved speeds for subsets that drop the ultra dynamic constructs (and usually a lot more dynamism and features to achieve that speed).

And the fact that there are languages as dynamic as Python is not as important as how it is implemented through the language; Lua is about as dynamic as Python, but with effects that are much more local in general. IIRC Mike Pall (of LuaJIT fame) said that Lua was almost optimal for JITting, JS is sort-of-ok if you ignore some things, but Python is beyond hope -- and I tend to think he knows what he's talking about, given that his hobby, one-man-show LuaJIT handily beat all well-funded-with-tens-of-employees JS engines for a very long time (maybe still does).

I wonder how good the SmallTalk JITs really are by today's standards, in the presence of e.g. "become:". The fact that's it is there, and supported, doesn't mean it's in common use, or supported efficiently -- just like JavaScript's "with".

In a sense it is, because in those communities the approach has been, "it is hard but we will solve it sooner or later", whereas in the Python world, PyPy seems to be the only approach that has embraced this culture, whereas everyone else takes the path "it is hard, Python cannot be made fast because it is dynamic".

That’s not been my impression. On the contrary - there have been very many unsuccessful attempts (Unladen swallow, Pyston, Nuitka, cython-for-plain-python, a few more I can’t recall right now); they all tried to maintain compatibility (complete language and binary api), and all delivered very meager speedups.

Those that focused on a subset (numba, shedskin, rpython) were actually able to deliver significant speedups. Even PyPy dropped binary compatibility.

The odd one out is Psyco, the precursor to PyPy. Perfect compatibility with non trivial speedup - but was considered a dead end by its authors, and that’s why they started PyPy

I wouldn't have thought `become:` was an example that distinguished a dynamic programming language from others.

Where have you seen it used as an example in that way?

It doesn't distinguish per se, it is rather a good example of something that a dynamic language can make that in a single call makes invalid all assumptions that JIT might have gathered until that point.

As for become: usefulness, Gilad Bracha explains it better than ever will, https://gbracha.blogspot.com/2009/07/miracle-of-become.html

He does not address JIT efficiency here. A likely implementation is just “lets drop all assumptions related to the type of A”. It would work, but frequent use would kill performance.

Python has all the issues that “become” has, with hundreds of “assumptions might have changed” points along common execution paths.

Liajit, for example, hoists everything it can outside of loops rather quickly, because it knows the cases “become” or Python equivalent can happen, and they are rare and uncommon in idiomatic Lua code.

Introspection is a gigantic issue.

Could you elaborate on that?

cpython's specific implementation is a "feature" that many things rely on, but even without that, python's semantics are very dynamic and it's hard to unbox anything in a way that is guaranteed not to be observable.

There are a few different classes of introspection. The first is commonly used builtins like “isinstance”, “callable”, “getattr/hasattr” etc.

The second is the inspect module that builds on that, allowing you to get the stack frames, inspect function signatures etc.

The third is trackbacks, which need the stack and themselves are introspectable.

The fourth is metaclasses (making dynamic classes at runtime).

These are heavily used in Python, and are optimisation barriers. For example PyPy (Python with a JIT) disables optimisations when “inspect” is imported or used anywhere in the call stack.

How many of these "levels" (not to confuse them with actual classes) exist in javascript? Only the first one?

(that said, I guess the fourth one (metaclasses) isn't a clear yes/no, since OO is very different in js)

Hard to compare exactly with that classification, but JavaScript only has the first one, and arguably parts of the fourth one.

metaclasses isn't really a major optimization hazard, it effectively boils down to hijacking the "class" syntax to instead run some custom code. It's used by things like dataclasses and the Django ORM to use class syntax to make a little DSL.

dataclasses do not use metaclasses (I’m the dataclasses author).

Apologies, I must have been confusing it with some other Python syntax tricks I've seen. I don't write too much Python anymore..

Smalltalk JITs do just fine with introspection and certainty Python does not have some magic feature that Smalltalk lacks.

Despite being a totally different language based around message passing with no control structures?

None of those are insurmountable, they merely make writing a JIT harder if you want to support them: it’s hard to inline a function call if the inclining can have a visible effect.

That is the whole point, it doesn't get more dynamic than that.

I’d say the lack of control flow structures and message passing would make it far, far easier to JIT.

If that was the case, there would be no need for StrongTalk.

Can you really not think of any other benefits?

I certainly can, and I can also think of several examples of dynamic languages that succeed where Python fails.

All of them just as dynamic, if not more.

If you can think of other benefits to Strongtalk other than “it makes jit compiling better” then why say that there is no reason for it to exist if Smalltalk’s design makes it easy to JIT?

Python is not even able to do the basic Smalltalk part, so it doesn't matter what StrongTalk brings into the picture.

Ok now we are on the same page about there being a need for Strongtalk other than “it may or may not JIT better” we will loop back a few comments and try again:

I’d say the lack of control flow structures and message passing would make it far, far easier to JIT.

But there are other dynamic languages with crazy introspection that are among the fastest dynamic languages there are. Common lisp is one example.

Yes, it's the GIL [1]

[1]: https://en.wikipedia.org/wiki/Global_interpreter_lock

https://mail.python.org/archives/list/python-dev@python.org/...

The GIL just prevents thread-level parallelism - which JavaScript also doesn’t have.

It depends on the runtime environment, no? For example, Node supports worker threads

https://nodejs.org/api/worker_threads.html

Worker threads are share-nothing, closer to processes than actual threads.

Worker threads aren't share-nothing, you can share data through SharedArrayBuffer which lets you share arbitrary binary data.

If they were share nothing, we probably wouldn't have shipped them.

Source: I'm Node.js core and one of the people who worked on them

You can also share arbitrary binary data between processes by using shared memory (e.g. Python has support for this in multiprocessing.shared_memory module).

Sorry, I realized after posting that I should have said "share-nothing by default". There are ways to share memory with a worker, but it's explicit and opt-in via a dedicated API. It's not like in Python where you can arbitrarily change global state from any thread.

Reply on news.ycombinator.com