shuss

There are many impediments to scaling vibe coding. We built a tool to internally scale vibe coding to vibe engineering: https://mfbt.ai/blog/vibe-coding-vs-vibe-engineering/

Fast changing libraries are a huge pain. That's why a no-code approach like Unstract (https://github.com/zipstack/unstract) makes sense.

There's also Gocker: https://github.com/shuveb/containers-the-hard-way

Yep. Docker is famously written Go. The only reason I wrote this is as an educational tool to discuss how containers under Linux really work. It’s self-contained (pun unintended) except for the usage of the go-containerregistry package, which anyway does stuff unrelated to how containers are managed on Linux.

This is such a great point. Never thought how async I/O could be a problem this way. In the SQ polling example, I used BPF to "prove" that the process does not make system calls:

https://unixism.net/loti/tutorial/sq_poll.html

Could be a good idea to use BPF to expose what io_uring is doing. Just a wild thought.

io_uring_get_probe() needs v5.6 at least.

True. Have to agree, here. Although one advantage over aio for block I/O that io_uring will still have is to use polling mode to almost completely avoid system calls.

The raw io_uring interface, once you ignore the boilerplate initialization code, is actually a super-simple interface to use. liburing is itself only a very thin wrapper on top of io_uring. I feel that if you ever used io_uring, after a while you'll end up with a bunch of convenience functions. liburing looks more like a collection of those functions to me today.

One place where a slightly high-level interface is provided by liburing is in the function io_uring_submit(). It determines among other things if there is a need to call the io_uring_enter() system call depending on whether you are in polling mode or not, for example. You can read more about it here:

https://unixism.net/loti/tutorial/sq_poll.html

Otherwise, at least at this time, liburing is a simple wrapper.

For regular files, aio works async only if they are opened in unbuffered mode. I think this is a huge limitation. io_uring on the other hand, can provide a uniform interface for all file descriptors whether they are sockets or regular files. This should be a decent win, IMO.

The author here: All examples in the guide are aimed at throwing light at the io_uring and liburing interfaces. They are not very useful or very real-worldish examples. The idea with this example in particular is to show the difference how readv/writev work synchronously vs how they would be "called" io_uring. May be I should call out the fact that these programs are more tuned towards explaining the io_uring interface a lot more in the text. Thanks for the feedback.

Oh, yeah. QEMU 5.0 already uses io_uring. In fact, it uses liburing. Check out the changelog: https://wiki.qemu.org/ChangeLog/5.0

First off, you're right that readv(), like read() can indeed return lesser data than was requested. Thanks. I'll fix that.

> I'm also actually not sure if in the uring version requests are guaranteed to be processed in order or not. I would have assumed the latter - and thereby the printed result could have been out of order.

io_uring may not make completions available in the same order as submissions. The article goes into some detail explaining this. However, the "cat" example only deals with one request at a time which uses readv() and readv()'s iovecs are returned in order. So, this is not a problem. The "cp" example does deal with multiple submissions and completions, however it uses writev() on the completion of each readv(), writing the correct offsets. So, it wouldn't matter if completions arrive out of order. But this is indeed something to look out for.

Yes. Please see io_uring_wait_cqe_timeout() in liburing.

Yes, you could do that. But I wanted to keep the example programs in the beginning very simple. I the same vein, they only deal with one request at a time, for instance.

Nope. io_uring is an API that helps with asynchronous I/O.

Weirdly no one had done it so far. That's the reason why I cooked up some high level devices :)

It it's not the file paradigm, it has to be system calls (plenty of them) set in stone, which could quickly turn into a problem or you probably mean a wrapper library?

The one point I keep driving is Kubernetes does so much useful stuff that is a pain to manage otherwise, that it's totally worth the risk currently posed by its rough edges. I'm hoping that came out clear in the article.

While already available tools should have sufficed and PSI should be good for it, a very important capability that PSI provides system administrators is its high-resolution. Using PSI, with specialized tooling, system administrators can automate sensitive workload scheduling.

I'm sure someone is already working on integrating information made available in PSI to Kubernetes so that the automation bit is taken care of.

kees99, you should check the "total" fields. They have information in microsecond resolution.

Thanks for sharing this. This is such a great resource.

I've never really done any serious work on anything other than Linux. But I always wanted to try out kqueue(). Even if I don't add a separate section on kqueue(), I think it warrants a clear mention.

OK, noted. Thanks for sharing.

That should be pretty trivial to implement with getrusage()

http://man7.org/linux/man-pages/man2/getrusage.2.html

OpenBSD and FreeBSD do have sendfile(), but args are different compared to the interface presented by Linux. It should be trivial to write a wrapper, though:

https://man.openbsd.org/FreeBSD-11.1/sendfile.2 https://linux.die.net/man/2/sendfile

String handling in C is such a PITA. Especially if you're used to languages like Python.

LOL! Thanks :)

Yes, IIRC, even Nginx uses the general structure you're describing.

Thanks for sharing!

Never seen this before. Very interesting. Thanks for sharing!