EdSchouten

It has virtually 0% unit testing coverage.

I wish someone would design a CPU where pointers refer to individual bits instead of 8-bit bytes. Sure, most instructions may still enforce that pointers are multiples of 8, but that’s no different from how existing instructions may have alignment requirements. I fail to see how such a design would be any less efficient if executed properly.

I think it’s silly that we always need to jump through hoops to declare and interact with bitmaps. Why can’t I just write the following?

bool my_bitmap[1024];

In an ideal world C++ wouldn’t need template specialization to make vector<bool> efficient.

Sacrificing three extra bits in every pointer also wouldn’t really lead to exhaustion of the address space. Intel and ARM CPUs don’t offer a full 64-bit address space anyway.

A "free distribution service and an open-access archive for nearly 2.4 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics."

No, in this case I have no interest in pursuing that.

Two months ago I decided to upload my very first paper to arxiv.org. As I'm not associated with any research institute, the process has been pretty painful. First I had to get endorsed by someone else to be able to submit a paper to a specific category (cs.DC in my case). After getting that sorted out I was able to upload. However, my submission has been "on hold" ever since.

I really wish arxiv.org provided a bit more transparency here. Why is my submission "on hold"? How many submissions are in front of me? What's the ETA?

The time complexities given in this article would be easier to read if they used |V| and |E| instead of m and n.

I guess it took a bit longer to get it adopted within Go because of some additional challenges:

https://go.dev/blog/swisstable#go-challenges

Interesting that it doesn’t use the same open source license as Vorbis/Opus itself. Otherwise improvements like these could be upstreamed?

> How does the monitoring system have any of the context to add labels? That would only exist in application memory.

Indeed. If you have a protocol that doesn’t allow exposing that kind of information, then that only lives in application memory. But my suggestion is that it’s exposed.

If I understand that correctly, it means your app always creates traces, and Grafana Cloud is responsible for sampling/aggregating. That may be prohibitively expensive in terms of CPU/network load.

What I’m suggesting is that your apps by default only send metrics to your monitoring system, but that the monitoring system can specifically ask to “upgrade” metrics to traces. Or to log entries.

The same thing with metric cardinality: by default, only report metrics in a fully aggregated manner. But do tell the monitoring system how they can potentially be broken up if needed (i.e., which labels to add).

What always puzzles me about OpenTelemetry is that tracing, metrics and logs are all designed independently. I wish there was a way I could just annotate my code base once, and let the ultimate decision to expose something as a metric/log/trace be dynamic at runtime.

For example, if I look at a graph in monitoring dashboard and see something suspicious, I’d like to say: “The next time something like this occurs again, please save me a trace.” I should be able to just do that with a single mouse click.

I remember them releasing the tracing spec/SDKs and saying “now let’s move on to metrics/logs.” That never sat right with me.

Go is often thought of as a successor of C. C doesn't have methods, only global functions. From my perspective, Go added methods primarily so that you can use them in combination with interfaces. Given that interfaces don't support generic methods, I'm personally not convinced that this feature was worth adding.

> For the mathematicians that created Computation Science, the only interesting solutions are complete solutions to general questions

There is a bunch of research devoted to Polynomial Time Approximation Schemes (PTAS). Mathematicians also take part in it.

The area on the map that is green corresponds to the blue banana: https://en.wikipedia.org/wiki/Blue_Banana

Yeah, that Sudoku puzzle has multiple valid solutions, whereas the page only seems to accept a single one.

It looks like the code of the course was 2WC16. Unfortunately the course material no longer seems to be available online.

I still remember following Andries’s “Linux kernel hacker’s hut” course he taught at the Eindhoven University of Technology (TU/e) back in 2010. Every week we’d get an assignment where we had to write exploits for commonly occurring security vulnerabilities (e.g., buffer overflows, bad printf format). It was one of the most enjoyable courses I ever followed. Thanks for that, Andries!

> If sizes are unsigned, like in C, C++, Rust and Zig – then it follows that anything involving indexing into data will need to either be all unsigned or require casts.

I don’t really get this claim. Indexing should just look up the element corresponding to the value provided. It’s easy to come up with semantics that are intuitive and sound, even if signed integers or ones smaller than size_t are used.

You can always use a divide and conquer strategy to compute the chunks. Chunk both halves of the file independently. Once that’s done, you redo the chunking around the midpoint of the file forward, until it starts to match the chunks obtained previously.

It depends on the architecture. On ARM64, SHA-256 tends to be faster than BLAKE3. The reasons being that most modern ARM64 CPUs have native SHA-256 instructions, and lack an equivalent of AVX-512.

Furthermore, if your input files are large enough that parallelizing across multiple cores makes sense, then it's generally better to change your data model to eliminate the existence of the large inputs altogether.

For example, Git is somewhat primitive in that every file is a single object. In retrospect it would have been smarter to decompose large files into chunks using a Content Defined Chunking (CDC) algorithm, and model large files as a manifest of chunks. That way you get better deduplication. The resulting chunks can then be hashed in parallel, using a single-threaded algorithm.

That’s great! People who do that are often inconsiderate of how it affect others. First of all, it generates unnecessary noise, which is annoying for neighbors who are still trying to sleep. Pedestrians/cyclists also need to breathe those exhaust gases.

I don’t understand why I would need to care about this. Can’t my operating system and/or pthread library sort this out by itself?

Yeah, that's true. Having some kind of chunking algorithm that's content/file format aware could make it work even better. For example, it makes a lot of sense to chunk source files at function/scope boundaries.

In my case I need to ensure that all producers of data use exactly the same algorithm, as I need to look up build cache results based on Merkle tree hashes. That's why I'm intentionally focusing on having algorithms that are not only easy to implement, but also easy to implement consistently. I think that MaxCDC implementation that I shared strikes a good balance in that regard.

In my case I observed a ~2% reduction in data storage when attempting to store and deduplicate various versions of the Linux kernel source tree (see link above). But that also includes the space needed to store the original version.

If we take that out of the equation and only measure the size of the additional chunks being transferred, it's a reduction of about 3.4%. So it's not an order of magnitude difference, but not bad for a relatively small change.

Yeah, GEAR hashing is simple enough that I haven't considered using anything else.

Regarding the RNG used to seed the GEAR table: I don't think it actually makes that much of a difference. You only use it once to generate 2 KB of data (256 64-bit constants). My suspicion is that using some nothing-up-my-sleeve numbers (e.g., the first 2048 binary digits of π) would work as well.

I’ve also been doing lots of experimenting with Content Defined Chunking since last year (for https://bonanza.build/). One of the things I discovered is that the most commonly used algorithm FastCDC (also used by this project) can be improved significantly by looking ahead. An implementation of that can be found here:

https://github.com/buildbarn/go-cdc

So 19494 is the largest? That's far lower than I expected. There's nobody out there that has put a date in a version number (e.g., 20250915)?

Soon available for people in the Vatican as well?

Exactly! At the same time you also don't want to call into the kernel's internal malloc() whenever a thread ends up blocking on a lock to allocate the data structures that are needed to keep track of queues of blocked threads for a given lock.

To prevent that, many operating systems allocate these 'queue objects' whenever threads are created and will attach a pointer to it from the thread object. Whenever a thread then stumbles upon a contended lock, it will effectively 'donate' this queue object to that lock, meaning that every lock having one or more waiters will have a linked list of 'queue objects' attached to it. When threads are woken up, they will each take one of those objects with them on the way out. But there's no guarantee that they will get their own queue object back; they may get shuffled! So by the time a thread terminates, it will free one of those objects, but that may not necessarily be the one it created.

I think the first operating system to use this method was Solaris. There they called these 'queue objects' turnstiles. The BSDs adopted the same approach, and kept the same name.

https://www.oreilly.com/library/view/solaristm-internals-cor...

https://www.bsdcan.org/2012/schedule/attachments/195_locking...

> Calls to bind(), connect(), listen(), and accept() can be used to initiate and accept connections in much the same way as with TCP, but then things diverge a bit. [...] The sendmsg() and recvmsg() system calls are used to carry out that setup

I wish the article explained why this approach was chosen, as opposed to adding a dedicated system call API that matches the semantics of QUIC.