Hacker News Reader

Comment by MichailP | Hacker News Reader

Can someone knowledgeable compare PyPy, Numba and Cython? I mostly use Cython. Tried Numba also, it has very nice workflow when it works with autojit (when it doesn't error messages are pretty cryptic). With PyPy I don't understand why for example all that type information obtained from jit wouldn't be used to make something like Numba specialized functions or Cython modules (noob question but please answer).

View on HN

Cython is a compiler not an interpreter. Numba requires explicit hinting and can only optimize some undefined (?) subset of Python.

PyPy on the other hand is simply a fast Python interpreter. Not sure what you mean regarding the type information gathered by the JIT.

For example does PyPy always warm up, or can it store "warmed up" version of some function (where type of objects is inferred)? I know that this is not in accordance with highly dynamic nature of Python, but not all functions are highly dynamic.

My understanding (from a previous HN discussion about javascript) is that since the output from a "just-in-time compiler" is actually machine code generated on the fly, it includes direct references to memory locations that are only valid for the lifetime of the process. So the output of a JIT is simply not in a format that can be saved and reloaded later.

Actually its possible to have a JIT like that but it is uncommon.

Do you mean can you save the state of the JIT for future use?

Yeah. Can you? Considering CPython already generates .pyc and checks timestamps, keeping some kind of JIT cache around wouldn't be a stretch.

No, the state of the JIT cannot be saved between executions.

I can't speak for python, but I do know that the impact of translating code for Racket's JIT (provided by GNU lightning, not LLVM) is negligable (ie. takes less time than the time you gain by JITting

Cython is not actually a compiler. It is essentially a more convenient way of writing C/C++ extension modules. It produces C/C++ glue code which is then compiled by an actual compiler like gcc. By tapping into the existing C API of cpython it is simpler than PyPy & NumPy which work on a lower level, but on the other hand Cython does little optimization for you.

Have you tried the cffi? I used to use cython, but since I found the cffi, I haven't looked back.

There are a lot of cffi c bindings. I used with success:

* https://github.com/amauryfa/lxml/tree/lxml-cffi

* https://github.com/chtd/psycopg2cffi

No, I got scared away by Simple example and Real example from the docs :) Do you maybe have some hints/links for cffi?

PyPy still doesn't work with Numpy, though they're working on it.

That's not quite right. Stock numpy doesn't install, and nobody's working on that as far as I know, but they've reimplemented part of the numpy API (at least the Python API, not the C one), and the subset they've implemented works fine. I have an application in production that uses it.

PyPy has it's own implementation of Numpy (called numpypy). It's not 100% complete but quiet functional.

http://morepypy.blogspot.com/search?q=numpypy

Numba is not Numpy.

Reply on news.ycombinator.com