The fact that this is all on one cpu seems like a problem, does it not?
Does someone have a mental model of how to take advantage of multiple cpus without creating a mess? If so, can you explain it like I'm 5 (or more like I'm 55 and tired)?
Does someone have a mental model of how to take advantage of multiple cpus without creating a mess? If so, can you explain it like I'm 5 (or more like I'm 55 and tired)?
Now different tools work differently for this. If you use multiprocessing, then you have no implicit shared state (you can use the filesystem or shm &c. to share state, but not much is shared by default). Clojure e.g. defaults to no mutability, so you can share state safely by default.
That covers concurrency. For parallelism, there are some useful structures to. In Common Lisp, I usually write a program single-threaded, then I profile, and add use lparallel[1] to parallelize the slow part. 99% of the time the slow part is a loop, so making it parallel is fairly straightforward. If it's a series of possibly interdependent calculations, then the calculations can usually be represented as a tree, and lparallel has a tool for that as well.
There is absolutely nothing lisp-specific in lparallel, any language with first-class functions and some form of thread-local variables could implement all that it does. It's not a silver bullet but it does solve the "My cores stopped getting faster, but I have more cores now" problem about 80% of the time.
For 99% of applications that means you take your favourite threadsafe queue implementation for your specific language and launch threads and only communicate this way. The messages should preferably be immutable, copies or at the very least marked readonly via const if copying is too expensive.
I'd recommend you to take a look at Erlang.
Also remember that there is no magic pixie dust. Functional programming languages are not inherently better than non functional programming languages at concurrency and parallelism. They merely leverage the "no multiple writer" principle by default. Immutable data is threadsafe primarily because it's guaranteed that there is only a single writer.
You typically have a cluster of machines, so you have to be able to run multiple instances of your application simultaneously to use all the processing power. And once you can run mutliple instances of the application in parallel you can as well run 16 instances of it on a 16-core box.
The upside is that you don't have two levels of granularity (threads, machines) but only a single one (processes) which makes both the coding and the ops as well as stuff like capacity planning much easier.
Don't start a thread per work item. Start worker threads and distribute work among them. Never ever block a GUI thread.
Always know which thread works on which data structures and don't even make more variables accessible in your worker threads. Keep the variables multiple threads can access to an absolute minimum and make sure access is always protected by a mutex.
If you repect these rules, threading is safe and fun.