Performance Impact of Removing an Out-of-Band Garbage Collector

Heh! sadly very true.

Many of those who do care about the p95s of an internal service often do it with a sense of chivalry that they have that 5% of the users back. The end user latency percentiles would be quite different from the service side latency, more so if its composed by a "Last_of" or a Max operator.

At times I feel tempted to write a library that offers primitives to glue microservice calls such as fan_out, retry, timeout etc but one that can reason about the percentiles of the output from the percentiles of the input.

BTW are statistical properties of latencies covered in distributed system courses ? I thought they were mostly about different kinds of guarantees, such as consensus, atleast-once, atmost-once etc etc.

With HTTP/2 you could have a separate control channel carrying a text/event-stream payload, with messages indicating when the server was ready.

With HTTP/1.x, you could abuse HTTP trailers to indicate readiness - the load balancer could read the response until the end of the body, send that to the client, then wait until the server sends a trailer to say it's ready for more work. Or the load balancer could follow every real request with a request to /pleaseBlockUntilYouAreReadyForMoreWork, and only send another real request once it got a reponse to that.

With RPC protocols that aren't HTTP, and don't have a flexible presentation layer where this would fit, then you would probably have to add explicit messages to the protocol, which would be a shame.

This write barrier design allows ROC to efficiently reclaim unpublished objects when a goroutine exits simply by resetting the current sweep pointer back to the initial sweep pointer.

If you use a disposable goroutine for a task, then when it exits, any memory it allocated but did not publish to other goroutines can be reclaimed without doing any further work. It's very much based on the idea that if you can put off collection to the end of a task, you can do it more efficiently, like OOBGC, but approached from a different angle.

I have no idea how well it actually works in practice. I've wanted something similar in Java for years, though!

It's somewhat mitigated by the reference counted global heap for passing large messages by reference but multi-threaded reference counting is extremely hard to do fast and AFAIK Erlang doesn't really try and do anything too complex.

But as a PM I have a different question. Which is the impact of the response time of Github on the user experience? 25% reduction on an average of 2 sec improves greatly the user experience, on a 200ms for a service that has basically no competition is just a marginal improvement, on a 50 ms average is not noticeable.

How is it distributed across multiple operations? Improving the latency of loading a diff is probably more important than the latency of approving a PR. In one case I'm trying to work, in the second I'm done with the work.

Now, my experience with GitHub is that it is already reasonably fast unless I'm doing something stupid. So the CPU utilization is a good metrics because 10% less CPU means 10% less server I need to pay and that goes to their bottom line (I don't know their economics to decide if it is substantial or not). The 25% reduction in latency is just icing on the cake...

I mean a reactive application needs three things:

- a fast network connection to persistence. If you can't talk quickly to your database, latency eats your response time no matter what you do.

- Maybe some LRU caches across requests, but caches inside an application tend to become a maintenance nightmare quickly. Stales reads are fun, and stalled cache writes are too.

- And beyond that, it should be possible to handle most request data by stack allocation/deallocation with references to a cache version if you have a cache with little overhead, depending on your security requirements.

I understand the ease of use and simplicity of a GC'd language, especially given the time GC'd languages came around big time. But I've always been wondering if you really need a GC if you think properly about request and object lifetimes.