Hacker News Reader

Comment by edgyquant | Hacker News Reader

I'd like to see these 10 lines that supposedly avoid the problems with the GIL.

It still hits the problems, but it can still be better than single-threaded. That's the point of his comment. People say the GIL is bad so they throw the baby out with the bathwater and no longer use threads, which isn't very well-reasoned.

Been a while since I've used Python, but as far as I remember the GIL only affects Python objects. So if you use Numpy for operations, you can avoid the GIL.

Can you give an example for your last sentence?

Here's a link: https://scipy-cookbook.readthedocs.io/items/ParallelProgramm... And a quote in italics below.

And, more importantly for us, while numpy is doing an array operation, python also releases the GIL. Thus if you tell one thread to do:

  print "%s %s %s %s and %s" %( ("spam",) *3 + ("eggs",) + ("spam",) )
  A = B + C
  print A

During the print operations and the % formatting operation, no other thread can execute. But during the A = B + C, another thread can run - and if you've written your code in a numpy style, much of the calculation will be done in a few array operations like A = B + C. Thus you can actually get a speedup from using multiple threads.

> as far as I remember the GIL only affects Python objects. So if you use Numpy for operations, you can avoid the GIL.

Correct.

Threads cannot help Python-heavy code; they can help I/O-heavy or native-heavy code (assuming native code releases the GIL).

Even without multithreading, numpy programming can give amazing speedups. It's the reason why data science Python isn't dead in the water.

> I'd like to see these 10 lines that supposedly avoid the problems with the GIL.

Spawn a thread and handle disk or socket IO requests there? You can get fancy and use a thread pool so maybe 15 lines of code.

Not sure why you used the word “supposedly”. Threads have been in the standard library for a very long time.

Summing lists != IO

>Threads have been in the standard library for a very long time.

Yes and still run the risk of running into GIL problems. You claim to have shown 10 lines to an interviewer that proved he was wrong about the GIL still being an issue and I've yet to see an example of it not being one when using pure python. Yes there are certain cases where you don't hit the GIL, no that doesn't mean it isn't still an issue when dealing with threads.

I just meant that one way to make threads "work" is to use IO, so I assumed the OP did a trick along those lines. Basically two threads can receive data or read from disk a the same time. Same thing can be accomplished with a select/epoll setup but threads would just be less lines of code.

Otherwise another trick is to identify a call which uses extensions or releases the GIL once it goes into C.

Reply on news.ycombinator.com