Hacker News Reader

Comment by jbert | Hacker News Reader

> Doing blocking operations inside a go is a anti-pattern.

Sorry - could you elaborate please?

In my understanding, in golang it's fine to block a goro with a blocking call, whether that be a blocking syscall (read/write/sleep) or a blocking read from/write to channel (perhaps multiplexed with select).

The scheduler will run other goros on the threadpool until the blocking call finishes. That's the "secret sauce" which allows you to write synchronous code in go without single-thread-per-goro (and requires a runtime to arbitrate the system calls and schedule goros)

View on HN

> In my understanding, in golang

The distinction is that it's not go, `go` blocks are macros doing a bunch of transformations. `<!` and `<!!` have the same purpose (get a value from a channel) but not the same context, `<!` can only be called inside a (go) block and will be transformed into a parking of the thread (yielding to the runner), whereas `<!!` works at a lower level and will block their whole thread.

> The scheduler will run other goros on the threadpool until the blocking call finishes.

`<!!` and the coroutine scheduler are not aware of one another, calling `<!!` from within a coroutine will block the scheduler. `<!!` (and `>!!`) exist specifically to be called from outside the scheduler (from outside a `go` block)

Likewise, `Thread/sleep` is not aware of the coroutine scheduler (it would have to be monkey-patched), calling it will also block the scheduler itself. core.async provides `timeout` for the purpose of parking a coroutine.

Thanks. I misunderstood the original comment to refer to golang code. The edited comment is clearer.

In core.async code inside (go) blocks are run via a fixed thread pool (a large one, but still fixed), when you use <! and >! the code after the put/take is turned into a callback that is attached to the channel. The thread is then released back to the pool. This allows for extremely lightweight async code. In addition, goes can actually be GC'd when they deadlock and can never continue (all references to the channel a go is taking from are dropped).

Code inside (thread) blocks run on a dedicated thread. This allows for long running or IO bound processes to execute without clogging the main thread pool. Thread blocks do not rewrite their bodies into a state machine (unlike go blocks) and as such must use <!! and >!!. Since these threads are dedicated they do not return the thread to the thread pool when blocking. This allows for slightly better performance at the cost of an entire JVM thread for the life of the block.

TL;DR - Use <! and >! with go. Use <!! and >!! with thread. Mixing the two can have very unexpected results. In fact, using <! inside a thread block will result in a runtime exception.

Thanks. I misunderstood your (original, unedited) comment to refer to a golang goro (go func()), not clojure go macro.

Yes - this seems to be a key difference of golang's approach. The runtime+scheduler conspire to make a blocking goro much cheaper than an OS thread.

I've enjoyed working with core.async once I understood these constraints. It would be nice for code to "know" whether it's within a go block or not. That would alleviate needing do-something! and do-something!! versions of some functions.

By "know" I mean something like an in-go earmuff var.

EDITED for clarification and formatting

> It would be nice for code to "know" whether it's within a go block or not.

Part of it obviously does since `<!` can't be called outside of a go block. And cljs has somewhat less of an issue there as it does not have `<!!` (since without extension the language has very little allowances for blocking)

Reply on news.ycombinator.com