Hacker News Reader

Comment by yummyfajitas | Hacker News Reader

When you write concurrent software, you typically build two kinds of components.

The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs.

For example, in my project Hobo, I've got an immutable, read-only index with procedural, deterministic methods. Then sitting in front of it I've got a TThreadPoolServer which handles multithreaded access.

Single threaded: https://github.com/stucchio/Hobo/blob/master/src/org/styloot...

Library handles multithreaded part: https://github.com/stucchio/Hobo/blob/master/src/org/styloot...

Another thing that will work for data where contention is rare is software transactional memory. Most programmers are already familiar with SQL transactions, I don't see a reason they can't use it in memory as well.

View on HN

"The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs."

This is a very good illustration of how concurrency potentially leaks out across an entire application. You can see here that your concerns for concurrency appears in the cogs. It is certainly true that you have provided a layer of abstraction but only for a certain concurrency model, and your cogs will reflect that in their implementation.

I would also note that I wouldn't claim that all concurrency problems pose these kinds of problem. Graphics pipelines being the easiest (and best abstracted) example. But in general for the day to day code we write, concurrency concerns have a habit of being very hard to tidy away.

For a really good treatment on the virtues and limitations of transactional memory there is a great conversation between Rich Hickey and Cliff Click on the clojure mailing list. I can't find the original, but here is a good cleaned up summary of the exchange.

http://blogs.azulsystems.com.sharedcopy.com/cliff/2008/05/97...

Concurrency does leak out and influence design decisions, most notably the API of the rest of the app. This is a given and there is nothing to be done about it.

But the point I'm making is that once the API is fixed you get to program more or less normally. Look at the code I linked to - it's all standard procedural java. Some smart guys at Facebook developed the concurrency system. I did some ordinary programming subject to one additional constraint (immutability).

I imagine this is how the future will look - top programmers build APIs, and the army of programmers will build many immutable/transactional/etc subcomponents. With language support (such as what Haskell/Clojure already give you) this will be very easy to enforce - IBM's army of guys who can't handle concurrency will stick to writing pure functions.

Yes, and the way I understood the article is that the problem--not for the next generation of chips but two generations down the line--is that cog architecture scales up to 10s or 100s of processors but not thousands due to synchronization delay. That I believe is what David Ungar is attempting to address in his research.

Thanks guys, this kind of discussion is why I open HN every day for the past few years.

> The first are the cogs.

Would appreciate it if people around here (and elsewhere) would stop acting like the word "cogs" is universally recognized, like "ram" or "dos". I have no clue what "cogs" is / are, and the search engines aren't helping to make me any wiser (or at least informed).

Cogs are the teeth on gears. Metaphorically, they are used to refer to one small part of a machine, which are not useful on their own, but in aggregate can get useful work done. The term is sometimes applied in a derogatory fashion to people, implying that they are nothing more than a replaceable cog in a large machine.

Reply on news.ycombinator.com