Time is not a synchronization primitive

The dynamic is: developers write the the test and it usually works on their machines. If it does not there is a chance that they figure out what's wrong and rewrite it in the proper way. Tests often work locally, where the CPU and disk are nearly 100% available to run them. Sometimes they fail on CI systems. Developers scratch their heads and attempt the easy fix of adding a sleep of 1 second. That almosts always work and it took 5 minutes, code, commit, test run. They know that it stinks but they have stuff to do. Every few months somebody attempts to remove some of those sleeps with mixed and often unsuccessful results. Integration tests are particularly nasty.

Go has excellent primitive for this, a channel:

    // Setup a channel. We'll send one and only one messsage over it, then close it.
    // The message will be either an error (if something went wrong)
    // or nil, if everything is fine and we can proceed.
    ready := make(chan error)

    go func() {
        // No matter how this goroutine exits, when it exits the channel will be closed
        defer close(ready)

        // Do whatever we must: open a socket, run a subprocess, or something else
        lis, err := net.Listen(...)
        if err != nil {
            // Let it be known we've failed
            ready <- err
            return
        }
        defer lis.Close()

        // Announce the "parent" goroutine we're ready, without errors
        ready <- nil
        ...
    }()

    // Here, we block and wait for our "child" goroutine to tell
    // us something or at least die and close the socket.
    // You can extend this and e.g. use `select` if you want to implement a timeout
    err, ok := <-ready
    if err != nil {
        return fmt.Errorf("failed to start listener goroutine: %w", err)
    } else if !ok {
        return errors.New("listener goroutine died unexpectedly")
    }

For similar reasons, we similarly can't keep making sleep-sort faster by scaling the sleeps down.

I am inspired by Smalltalk but with a twist: it is the pattern and sequence of interactions (such as events, states or messages) between MULTIPLE objects - all interacting - that are interesting.

Golang's and occams select operator is powerful for switching between events from different sources.

My idea: I want to be able to efficiently epoll_wait/uring a sequence of steps and state changes between multiple objects.

The idea is complex event processing for POJOs or objects and states.

Epoll for general purpose objects has usecases in efficient and scalable change detection in microservices, graphical user interfaces dirtychecking, job systems and workflow systems.

If you have a pull or push driven system and you want to adapt it, your hand might be forced into a substandard solution to add reactivity to an event or state or event sequence.

In integration testing, I see this problem. You can fire events in Javascript that your test runner can listen to, this is one solution to this problem but you have to build it yourself. You could hook into Redux state management. I have only created APIs for pages for interrogation for use by Selenium. But what I feel you want is state communication that isn't dependent on timing but efficient notification. (I don't want to use Selenium ExpectedConditions Wait for which is what I generally used to fix tests, there's a far more elegant solution)

How do you arrange a date with your friend in the city? I usually say "let's meet at 17:00 at the main train station, under the clock".[0]

You have to know your counter-party's accuracy. But if you synchronize on any other metric, surely that could also be inaccurate? So how is time any different?

[0]: Note: in Danish: https://www.berlingske.dk/under-uret-uret-skiftes-ud

They do say "you generally want to use something that synchronizes that state", but then fail to describe exactly which method they selected in this particular example. A single sentence explaining the particular change would've been sufficient.

This is a little thing but would greatly improve the communicability of the post and practices greater empathy for the reader.

Time is fine for synchronizing when all components are actually reading and acting in accordance with a clock, and agree on using the same one (or a well-characterized facsimile of the same one).

Time is used at the bottom layers of hardware. Certain components generate a signal, and expect the response signal to be settled within a certain time.

Even in situations in which there is robust signaling (like a device can indicate for indefinite periods that it is "not ready" and another device will wait), the underlying signals on which that is based have to meet timing constraints.

You have rules like "when pin X is driven high, the data is expected to be present on lines Y_0 through Y_32 within time T_x." If not, there will be garbage.

All notion order in the machine rests on arbitrary timing at the lowest electronic design level.

- The general point is correct -- time is not a synchronization mechanism

- Flakey tests are bad for all the reasons listed

- Shoddy synchronization makes for flakey tests

The example chosen -- waiting for a test server to be available is probably a huge source of the issues. However, I don't think the example -- really demonstrates how to do synchronization.

My core objection is that it's really relying on the operating system to take care of the synchronization, and that winds up hiding it entirely:

- the dial will not happen until the operating system has competed the `listen` call

- the operating system is selecting a random port at the time of the `listen` call, and reports the chosen port to the user

This is the correct way to approach this problem, but it doesn't really show a proper synchronization. It shows the operating system synchronizing on behalf of the program. It's also an example that probably only shows up in test -- client and server in the same process space.

In practice, many clients and servers may start at the same time in different processes or on different hosts, meaning that the client has no signal at all that the server has become ready. The best thing that can really be done is quickly polling with low timeouts until the server reports healthy.

I'd have liked to see use of a `sync.Cond`, or channel to signal that the operation has completed, and the waiting goroutine being safe to continue. It wouldn't fit well in this example, because, well, the operating system provides those facilities.

Over, because it does come up a lot with the example used, I'm happy there is a blog posts addresses it. Hopefully I'll see less of the problem in the world. I suppose I hope the author writes a part 2, showing other ways sleeps creep in and how to remove them.

idk how common that is these days, but I saw it all over, used basically blindly to "fix" all manner of mystery reliability problems and never not one of those countless examples was that ever actually the problem or the fix. It was always simply failing to close a file before trying to do something else with it from some other context like a system() command. It was used mystic-dead-chicken style even worse that sticking sleeps everywhere for no actually understood reason.

either something systematic or a just a simple stochastic scheduler.