Thread pools: How do I use them? | Hacker News Reader

Thread pools: How do I use them?

jvns.ca

188 pointsingve45 comments

I like the style of writing in this post. Its fresh in the programming world where everyone seems to know it all for someone to admit something is hard or confusing.

Heck yea. Im kind of new here and it can be so intimidating. I feel like everyone is an expert at everything.

It's not that, it's just that experts tend to contribute to threads where they can share their expertise.

So while it appears that everyone here is amazing; that isn't the case: we just tend to have something that we're very good at and comment on it.

The good thing is that you can learn a lot here as long as you don't get sidelined by some of the inane arguments and fighting :)

Also, an awful lot of people who sound like infallible "experts" in tech are just people who are dealing poorly with imposter syndrome.

Hang on; Imposter Syndrome would mean that you would come across as a fallible, non-expert, since you believe yourself to really be an imposter. Do you really mean something like the Dunning-Kruger effect, which is sort of the opposite of Imposter Syndrome, people with little knowledge believing themselves to be infallible experts?

Impostor syndrome means that the person thinks of themselves as a fraud, not that other people do. So someone dealing poorly with impostor syndrome might well only chime in when they are exceedingly confident, or only when they do research and can be certain that the information/position they are providing is correct. If you keep your mouth shut except when you're 100% certain, you'll likely look like an "infallible expert" because you're always correct from an external perspective, even if you are quietly filled with self-doubt.

Pretty much. Some people who lack self-confidence seem afraid to ever admit "I don't know" lest it somehow reveal their "fraud" to the world.

A lot of people have discovered that speaking confidently can be a solid substitute for actual expertise.

They're called "consultants"

This is how everybody feels at one point or another. It's called imposter syndrome[0] and it's extremely common!

[0] https://en.wikipedia.org/wiki/Impostor_syndrome

Don't be intimidated. I've got 30 years of being in the business of software development, telecoms, networking etc under my belt and every day there's still something new to learn, sometimes too many new things :) Also don't be frightened to ask about things you don't understand, I find the folks around here are generally quite an affable bunch.

The trick is figuring out what you don't know to ask the question in the first place. Often not trivial.

It helps to try to tackle a project that you don't already know how to accomplish. Then the Q&A can work like a reverse dictionary. You know the answer, and work backwards to find the question. Every stumbling block is a question to ask.

Heh. Keep in mind that people are going to be more likely to comment on something which they have knowledge of or an opinion on, so you will see a certain amount of selection bias in that most of the people who comment on something know something about it, even if there are large numbers of people reading and interested who are just keeping quiet (though of course there are people who also make comments on things they know little about).

Also, I know that for some things that I write about, I'm not actually an expert on it beforehand. Instead, I see an interesting question, say to myself "hmm, that's interesting, and I don't actually know the answer", go out to Google, books I own, or just try some experimentation out myself, and then comment on what I found. I usually do try to provide sources and some details about what I did when doing this, but I realize that it may sometimes make me look like more of an expert on topics than I actually am. For example, I have a silver badge in JavaScript on StackOverflow, even though I've only ever written a few dozen lines of JavaSript in my life (in fact, I've probably written more lines of it on StackOverflow than off), since I tend to be good at quickly researching and experimenting with a question and providing a good explanation, not because I'm an expert on JavaScript.

So yeah, it's great the Julia writes down her thought and search process while doing this, as it helps people realize that they too can go through the same process and learn new things.

All of the posts on this blog are like this, so you should read the archives, for sure. I agree completely.

If you, like, don't mind the, like, valley/bro/speak lingo. AWESOME, right?

I like structured arguments against a status quo - that's what's really required here. This article is one notch earlier in the mental digestion chain; a kind of experiential report, an extended anecdote. It's harder to tell if the problem is the library or the user. It asks the reader to work more, because they need to go on a journey, rather than just parse an argument.

I enjoy reported experience too, but I have less time for them unless they're specific to something I have a strong interest in. Otherwise, give me the executive summary.

Yes, I read less of the internet than I used to.

I've read a lot worse on that score. At least the author didn't parrot a bunch of memes, or throw animated gifs everywhere.

Yes, the narrative form gives the impression that the author is a very! excited! person! full! of! energy!

But that's kind of refreshing sometimes. It's easy to get cynical about software engineering like I am. And I followed her stream of consciousness a lot better than I follow a lot of recondite technical prose, particularly when it concerns things mired in proprietary abstractions (Java).

And if you had actually met Julia (I am lucky enough to work with her) you would realize that this impression is 100% correct. It is not just an impression, but her whole approach to technology.

While it is good to have more cynical people too (especially for things like software stability), a company that forgets enthusiasm ends up losing its soul.

I think a perfect team would be one Julia, three or four cynical types, and maybe one fresh faced newbie to ask the dumb questions that nevertheless sometimes lead to something innovative.

According to her bio, she works in Stripe's machine learning department so clearly a smart gal and appears to have a great amount of humility - enjoyed her post, was easy to digest and a light read.

I have been following her for well over an year now for exactly this reason. She never preaches. It's more like taking her readers along on this journey of discovery. Very very humble and yet insightful

While i admit it was a thrilling take on a concept as abstract as thread-pools, i think the issue lies in the nature in which we programmers talk about computing theory.90% of the time we are either telling (stack-overflow help, or receiving (asking for help on stack overflow) which requires us either to be the teacher or the learner, never in between like this author is.

> Or maybe there is a totally simple way and this could take me 5 minutes!

At that paragraph I immediately thought of http://aadrake.com/command-line-tools-can-be-235x-faster-tha...

Holy shit.

> I have been in technology roles for over 17 years

I guess this is what experience gets you, huh.

Or if you know some basic performance characteristics of your machine (i.e. memory bandwidth + processor speed + number of cores) and roughly how much work you need to do (i.e. number cycles to do X), some back of the envelope calculations can often give you the same insight (i.e. "is X feasible with compute resource Y?").

I recently ran into the issue that the multiprocessing.pool.ThreadPool class is not cancellable in Python 2. It seems that only in Python 3 can you exit a ThreadPool with eg. a KeyboardInterrupt before all tasks are finished.

Since then I've been using this code, which works great across all platforms and Python versions: https://github.com/metachris/pdfx/blob/master/pdfx/threadpoo...

I've experienced this bug in Python 2 using "pool.map" and "pool.apply", but KeyboardInterrupt does work if you're waiting for a finite amount of time. So as a workaround you can use "result = pool.map_async(...); result.get(sys.maxint)". A little hacky but functional.

As is sometimes the case, people start with the wrong abstraction (thread pool) and then optimize it.

Java/Scala streams look like they would solve most of the problems in the post on their own.

e.g. Java streams

http://radar.oreilly.com/2015/02/java-8-streams-api-and-para...

    private void runParallel() {
      trades
      .stream()
      .parallel().forEach(t->doSomething(t) );
    }

Java/Scala has a hierarchy of concurrency abstractions, from high level to low level:

Streams, Parallel collections, Futures, Actors, Fork/Join, ExecutionContext/ExecutorService, Threadpools, Threads, synchronized, Barriers, Atomic values and many more.

Something not so obvious for beginners: Java/Scala does not use all the available memory. If you get out of memory problems with the 10gb data set and you have much more RAM available, you need to tell Java/Scala to increase the Heap.

My favorite blogger of 2016 thus far. Every post is great.

Also the bit that rings home the strongest is just how difficult it is to do trivial things. Even a senior level programmer with ten years of experience has to sit and say, why is this such a pain in the ass?

I guess that's one reason I really like clojure. Core.async has a really hideous API at first glance but using channels and buffers makes doing this sort of thing a lot more enjoyable. Anyway the intent of the article was to dive into abstractions not to suggest more of them so I'll shush.

Maybe "Scala concurrency surprises" would be a better title.

It uses the JVM's threadpools directly so it applies to more or less any JVM-based language, and the problems it talks about (expensive sequential operation before threadpool submission and unbounded threadpool queue) are pretty common issues.

Witness the unbounded and unconfigurable (though replaceable — at your own risk since you're setting a "private" variable of the pool) queue of Python 3's threadpool: https://hg.python.org/cpython/file/default/Lib/concurrent/fu...

Great writing.

I wonder if a ForkJoinPool [1] would be helpful here as it does work-stealing giving better utilisation.

Although I've read admonitions that they should not be used when doing I/O. [2] That article seems a little ranty and overblown though, so I'd like opinions on that.

1. https://docs.oracle.com/javase/8/docs/api/java/util/concurre... 2. http://coopsoft.com/ar/CalamityArticle.html

A coworker showed me Akka Streams recently. It allows you to overcome the impedence mismatch between threadpools processing jobs at different speeds where one task feeds into another. Could be a neat way to join up the i/o bound tasks to the cpu bound tasks.

https://opencredo.com/introduction-to-akka-streams-getting-s...

Yep, it's a really cool library. I'm pretty new to it myself, but I made a little gist of how one might approach this problem: https://gist.github.com/kevinavery/941e7d67c4f8b104f610

I was kinda surprised by how tricky it was to make a Flow that turned ByteStrings into separate String lines, but it seems like the custom GraphStage approach generalizes really well to more complicated stages.

https://rnduja.github.io/2016/03/25/a_first_look_to_akka_str... is also pretty good.

The article is definitely super ranty and a bit overblown. The Fork/Join framework is actually pretty solid and works for a wide variety of use cases and has a relatively simple API. It's an excellent article though and I highly recommend anyone implementing the use of Fork/Join to read it and get more in depth knowledge.

Java 8 made some improvements in code quality and safety and the framework underlies all of the parallel streaming abstractions used going forward.

It's definitely worth learning all of the concurrency abstractions available in the JDK too though!

> In my case, I was reading a bunch of data off disk. maybe 10GB of data. And I was submitting all of that data into the ExecutorService work queue. Unsurprisingly, the queue exploded and crashed my program.

In the spirit of brainstorming:

1) Abstract out the I/O step so that you can pass around a bunch of lightweight and lazy "I'll get the lines for you when you ask" objects. Then you can safely queue up a large number of them.

2) Choose to break the "job" apart into two parts, and submit the "get lines from filename" jobs into a different, smaller pool, with the results flowing into the larger (CPU-core-limited) pool.

The easiest way to use thread pools in Scala is to use an ExecutionContext. Akka will let you define an ExecutionContext from a configuration file using a wrapper called a Dispatcher -- you can ramp up the parallelism-max setting for more cores. Then you do actorSystem.dispatchers.lookup("my-dispatcher") and pass it to all the actor based code that you are using.

You can use a work pulling set of actors to handle computation after that, or use Akka Streams to provide backpressure, to avoid the OOM problem.

Always enjoy reading her posts, I particularly like her enthusiastic style too.

> Here's what that looks like in Python.

Although python multithreading outside of IO tasks or dropping into C code that drops the GIL isn't going to parallelize, the latter of which is the only way you're going to burn up cores. The API does make it seem simple though.

The example is using multiprocessing.pool.ThreadPool which does not have the GIL problem you mention (at the cost of making it harder to share data).

ThreadPool would lock the GIL, wouldn't it? I think multiprocessing.Pool uses processes and that's where you can avoid the GIL (each process gets its own) at the cost of making data sharing harder.

Even with the GIL, ThreadPools are still fine performance-wise for tasks that release the GIL such as I/O or a number of functions in numpy and scipy.

You're thinking multiprocessing.Pool. I think ThreadPool actually uses threads and does hit the GIL.

Good write up.

Reply on news.ycombinator.com