Hacker News Reader

Comment by coldtea | Hacker News Reader

Haven't seen this discussed on HN, and seems like huge news.

Why isn't this more known? Is it moving forward? Has it been announced officially...

I personally think it's a bit premature to talk [edit: for the developers to talk] about it - they're attributing 2.25x to the addition of a JIT, which generally takes a long time to develop and tweak, adds a significant complexity, and makes the performance profile complex (ie. slow start, slow first requests, etc.).

I like though, to see the JIT wave across interpreted languages (see Ruby). I'm also curious to see what's going to be the implementation.

The problem with this and all other efforts for speed in CPython is the interop with C extensions. For JITted Python look at PyPy, sure the JIT gives you some performance, but they broke all C extensions along the way.

> they broke all C extensions along the way.

It doesn't have to be this way though?

Many C extensions work with PyPy now. I have a service that uses pylibmc and pycurl, and it works with PyPy. Those extensions didn't rewrite with cffi or anything like that, PyPy added a compatibility layer, it's impressive.

But, you don't get the JIT performance if you use the C extensions a lot, it's better for PyPy performance if you find a pure-python alternative for extensions used in "hot paths".

The TruffleRuby approach is to run the C extensions in an interpreter with a JIT. This way you can change the C extension interface without changing the code, and the optimisations apply to both at the same time.

https://www.youtube.com/watch?v=YLtjkP9bD_U

Shouldn't a well-designed C extension interface work fine regardless of whether a JIT is used? Java's JNI interface, for instance, works fine with or without a JIT. Is it just that the interface was designed without JIT in mind?

> Shouldn't a well-designed C extension interface work fine...

Yes!

But we aren't working with one of those.

I personally think tracing JIT is a bad technology. It's over complicated and makes it hard to reason about runtime space and time efficiencies of code.

I don't know the best route for python to get more performance but it probably involves allowing people to opt out of dynamism and basically doing what cython does but for all possible code.

You're basically talking about mypyc: https://github.com/python/mypy/tree/master/mypyc

>I personally think tracing JIT is a bad technology. It's over complicated and makes it hard to reason about runtime space and time efficiencies of code.

Hasn't Strongtalk, Java, and JS put those concerns to sleep?

Also LuaJit.

But just the most basic optimizations as done in PHP, a good lisp, scheme or lua would reach the 5x goal.

No, trying to benchmark Java and JS are the reason for the concerns.

None of the current Java or JS engines use tracing JITs, though?

Is there a precise difference between how openjdk works where it needs a warmup to begin running performantly and how a tracing jit works?

If there a precise difference between how Ignition and Turbofan interact that is not a tracing jit? I really thought both Java and JS implementations were tracing jits. Because if I'm going to whinge about technology I don't like I want to whinge about the precise problem. :)

Yeah, if we only had closer speed to Java and JS for CPython, I'd have no concern...

>I personally think it's a bit premature to talk about it

Has that stopped HN before? We have devoted several top posts to the V language, LightTable, and several clearly doomed-to-fail efforts. This, in comparison is rather tame, from a legitimate source, and even if it just gives 2x performance, that will still be something.

PHP, for one, pulled it off, with PHP7.

I meant on the developer's side, rather than for the public to discuss it. I've edited the post to make it clearer.

Is there a name for the idea of "adaptive JIT"?

where the code starts executing without JIT, while another thread is instantiated that is doing JITting in the background, and as soon as the JITting is done, JITted version takes over seamlessly for the rest of the execution.

> Is there a name for the idea of "adaptive JIT"?

Some people say 'dynamic JIT' (for example HotSpot) as opposed to a 'static JIT' (for example .NET.)

Most modern JITs have several tiers like that...

Because it is on the idea stage without funding.

My guess is that it will stay that way. That said, it is probably feasible to do some parts of it, similar to the 3x3 project Ruby has. I wouldn't be surprised if numpy glue is a lot more tractable for optimization than Rails, but the interaction with C libs would pose some serious issues.

Maybe because the repo is 2 hours old, provides no actual information other than some hand waving. Also the author asks for $2 Million and doesn't seem to be a Python Core Developer. Given the repos age I'm probably answering to Mark Shannon himself.

Edit: This post wasn't supposed to be an hostile attack against anybody. Just found it odd to ask why an 2 hour old repo wasn't more known. I did not know that Mark Shannon is a core dev. Certainly did not pick that up from the repos contents.

I think Mark Shannon is a Python core dev now. But I agree with your other reasons.

Mark Shannon definitely is a core developer, though I agree that it's odd to ask "why isn't this more known?" about a two hours old repo.

He is a core developer (and long time core contributor), and he doesn't ask for "$2 million dollars".

In fact the post says: "The PSF seems to be the obvious organization to coordinate funding".

And no, I'm not Mark, somebody send me the link. Conspiracy theory much?

The repository linked mentions that each stage should cost about $500k and there are 4 stages. You can argue that he is not asking for it, but at least he is valuating it at $2M.

Didn't say he isn't valuating it at $2M. It's all on the post. What he doesn't do is say he should be the one to handle those (or even get those).

200k per year would not be the worst salary to pay a developer for this work, and I'd say he's outlined about 5 to 10 years of work. Feels like you could negotiate a package of escalating compensation contingent on delivery milestones.

Reply on news.ycombinator.com