Comment by rubyn00bie | Hacker News Reader

Comment by rubyn00bie | Hacker News Reader

Parent Full thread

I guess I wish this was a bit more in-depth as to what "go" can do with channels or goroutines, but maybe I've just been using languages where all this is already possible. The article is a nice cursory glance, I just want to learn more :)

I mean the first example it looks like is using a Mutex, and then (b)locking on it, and the second just looks like having a queue of messages (mailbox) that it rips through (like the actor pattern).

Some questions I've got after reading this...

How does the go-runtime (?) schedule these calls? Does it manage an internal thread pool? Is the scheduler asynchronous, parallel, or both? How do you manage contention or back pressure if I begin to flood messages to to one channel (or many)? How many channels can I have open and what are the limits? Can I still lock the system stupidly using channels, if so, how (or how not)?

Edit: Truly, I'm curious because as I researched asynchronous programming and efforts to better future proof my career (years ago) as we began really increasing core counts-- Go never stood out. It's a fairly practical language, yes, but if I want a better paradigm for asynchronous programming the future it really isn't there (IMHO). BEAM stood out as something unique, the JVM stood out as something even more practical, and Rust stood out as something performant (with the serious caveat of not being C or C++), while Go has always seemed like an odd one to me... People talk about channels and goroutines like their special but they seem pretty damn run of the mill to me... WAT AM I MISSING?

Goroutines are scheduled onto system threads via M:N greenthreading. Each "system" thread can steal work from other threads, and there's some stuff so that blocking system calls don't use up your "goroutine system threads" (you can put a limit on the number of goroutines that run concurrently). Channels are fixed size on creation, either 0 (directly handing off items from 1 goroutine to another) or >0 (the channel has space allocated to hold that many items). If the channel is full(buffered, i.e. >0 size channel) or there's nothing waiting to receive the item(unbuffered channel), the sender sleeps until it can send the item. Channels are purely data structure, have as many as you want. You can lock the system fairly trivially, but Go can detect it in many cases and give you stack traces of all the places stuff is waiting to read/write (only if all goroutines deadlock).

edit: And no, there's nothing really special about channels, just they play nice with the goroutine scheduler, so it's perfectly sensible to do lots of "wait on these channels" stuff inside your program, without having to have lots of OS threads and such. (goroutines are a bit more lightweight then OS threads)

Go channels have a fixed capacity, by default zero, and their main special feature (compared to just having some concurrent_queue<T> type) is the select statement, which helps devs do things the right way. Nothing stops you from stupidly locking the system, e.g. by having a thread block itself by writing to the channel it’s responsible for consuming.

Is the select semantically different than something like this?

    bool TryWrite(T)

How does go handle cancellation with the select syntax? I guess you have a cancel channel and select across both the cancel and the read channels? Is Go smart enough to sleep and not busy wait in that case?

> Is the select semantically different than something like this?

Yes. select can do a lot of things. It's overloaded to be something like 5 different things. Here, let me show you:

    chanOne := make(chan int)
    chanTwo := make(chan int)
    ctx := context.Background()

    // Receive whichever is ready first. Blocking
    select {
    case v := <-chanOne:
    case v := <-chanTwo:
    }

    // Non-blocking read
    select {
    case v := <-chanOne:
    default:
    }

    // Timeout and cancellation
    select {
    case v := <-chanOne:
    case <-time.After(1 * time.Second):
    // timeout
    case <-ctx.Done():
    // cancel
    }

    // Write or read, whichever channel is ready first
    select {
    case chanOne <- 1:
    case v := <-chanTwo:
    }

Another form of cancellation is closing channels, which causes reads to return the zero value immediately, and causes any writes to that closed channel to immediately panic (effectively abort the program).

Neat. Most of the permutations are easily handled in other languages as well except for the full blocking (sleeping) case. Without a channel the scheduler can reason about, most languages would have to busy wait or possibly pull from multiple channels.

I think the biggest advantage is that it's a lot harder to screw up using a select statement than most equivalent API's you'd create in other languages that allow waiting on one of N channels/signals/channel/event operations, and branching based off that.

Simply because you're either constructing a bunch of callback objects or you get some number N saying the Nth parameter had an event, and your code has to match N to the specific channel correctly.

Go does put that goroutine to sleep, freeing the underlying OS thread to work on other goroutines, so it's very lightweight.

It's like what you'd do with actors on top of some kind of userspace scheduling (e.g. a thread pool that accepts tasks), yes. Writes to channels are implicitly yield points, so you write cooperative multitasking code (again, like you'd do in any language with userspace scheduling) with all the advantages that entails (no blocking), but without having to explicitly manage your sequencing/yielding. Since the whole language is built around the idea that this is going to be happening, a lot of the immediate problems with userspace scheduling go away: anything that's trying to do thread-local storage (e.g. web requests or database sessions getting bound to the current thread) will necessarily be goroutine-aware and do the right thing.

There's no free lunch, of course: it becomes much harder to write code that is pinned to a single system thread (e.g. good luck interoperating with a C library that expects you to pass callbacks), or to use higher-performance blocking I/O in the cases where that's warranted. But it makes the 80% case very straightforward.

https://www.youtube.com/watch?v=h0s8CWpIKdg

Reply on news.ycombinator.com