Python newthreading - simplified threads, path to eliminate GIL | Hacker News Reader

Python newthreading - simplified threads, path to eliminate GIL

mail.python.org

80 pointstimf18 comments

I know it's not the same thing, but I quite liked multiprocessing. Although it's severely limited in terms of passing objects around and there is always the overhead of spawning new processes.

As a sidenote, the John Nagle there is the same one in Nagle's algorithm: http://en.wikipedia.org/wiki/Nagle%27s_algorithm

Good to hear (thanks)! Alas, multiprocessing will always be slagged by the serialization and deserialization of objects. Note, I'm the maintainer, so I'm not trolling :)

Sheesh, all these years you think people who write core modules are robots and then one day one comes up and replies to you on a message board! :) What you say is true, but in any case, my congratulations on a very pretty API.

He might still be a robot. :o)

Especially in the original sense of hard-working "biological entities that can be mistaken for humans" ;-) http://en.wikipedia.org/wiki/R.U.R._(Rossum%27s_Universal_Ro...

Incidentally, another story by Karel Čapek - http://en.wikipedia.org/wiki/The_Absolute_at_Large - features a "reactor that can annihilate matter to produce cheap and abundant energy [and] as a by-product, the absolute. The absolute is a spiritual essence".

I immensely enjoy "War with the Newts" (http://en.wikipedia.org/wiki/War_with_the_Newts) and only realized after reading the Wikipedia page, that it is by the same author, Karel Čapek.

There are dozens of things that would have to be broken to make this work, for instance sys.set_trace. Either tracing is broken, or tracing is rewritten with a new threading strategy (that's not this strategy) or you're back to GIL-land.

Nothing is impossible - not even GIL removal. Just very hard.

Some things may be as hard as a complete rewrite, though.

When talking about an interpreter; I find those terms completely acceptable. So long as the language itself continues to work, and I can still use C extensions.

The C extension are one reason why some features are so hard to implement. E.g. reference counting will always be with us, because the current interface to the C extensions rely on it. And you had to change all of them, if you wanted to get rid of it.

Very, very interesting. I'm going to have to suck it up and pull it from sourceforge - I'll be very interested to see how they implemented the message passing and "safe object" stuff. With time this could change things quite a bit for python.

The path from a pure python user-level threading library to eliminating the GIL in CPython is longer than the subject implies.

Something to keep in mind with Python/Perl/Ruby anyway: Making your app run on 2 cores makes it 2x faster. Implenting your algorithm in Haskell instead makes it 50x faster.

Cooperative userspace threads work very well for the kinds of applications you write in Python.

What is with developers naming their projects newsomething?

Django had newforms, Java had nio and Guido van Rossum just posted about the multiple inheritence method resolution order (MRO) which had a python 2.2 new-style and python 2.3 new-style. I could probably come up with a huge list of newwhatever projects.

It's such a lazy, uninformative, short-sighted way of naming your work.

Naming things 'new-' something is not IT developer specific. New College, Oxford was founded in 1379.

That's true, there's also The New School in NYC.

Someone voted me down; I suppose I'm am being a little too aggressive about it but I do think it's a stupid way to name a project. :-)

Most of these projects represent a lot of work and a huge revolutionary leap over the "old" projects. It seems like that much work deserves a more fitting name.

"There are only two hard things in Computer Science: cache invalidation and naming things." -Phil Karlton

Looks very cool.

Of course, I am already fantasizing about it being merged to the main branch. Assuming it will work well, is this something that can get into 3.2?

I saw this release schedule for 3.2: http://www.python.org/dev/peps/pep-0392/

I see that 3.2 alpha 1 is due today.

No, it would not get into 3.2, and it doesn't offer speed improvements at this time. It's a great experiment right now. It's probably a year+ off from ever seeing core, if ever.

Reply on news.ycombinator.com