Hacker News Reader

Comment by dvdplm | Hacker News Reader

So what would be the reasons to run pg as a unikernel? To get max performance on bare metal? Smaller attack surface? Or?

View on HN

The less between your application and the hardware the better you can utilise it. There’d be basically no contention for resources, IO could go basically straight to the drive with less fsync-shenanigans etc.

But is the impact really noticeable? I believe it would then we would have seen more ready and stable solutions.

With our Linux-based unikernel you can make small adjustments to the application -- usually just a handful of lines of code -- and get big performance improvements, like 25% or so. We had a paper about this at Eurosys'23: https://dl.acm.org/doi/abs/10.1145/3552326.3587458

That‘s the problem of unikernels. There are papers how great they are but almost nobody is using them. If applications could easily get a 25% boost without much work they would do it.

But nobody is using it. So I am not trusting the „benefits“.

There are a ton of well known techniques in computer science that will make software faster, more reliable or better in other ways, that people don't use, yet really work.

I mean, if I had an easy-enough way to use them, I absolutely would.

Docker isn’t ideal, but it’s common, and the UX is decent enough. If a unikernel solution existed with near-enough quality of UX and deployment, I’d 100% switch.

> But nobody is using it.

The development community at large often chooses to use/not-use things (packaging solutions, tooling, engineering approaches, software architectures, etc) for a huge variety of reasons, not all of them based in any kind of logic or benefit. I suspect microkernels could become a viable solution for some things, they just need a “docker moment”.

If you look at PostgreSQL configuration settings, you'll notice a chunk of them are dedicated to dealing with fsync. They even have a dedicated testing tool to test how the database behaves with different modes of syncing.

fsync is a serious performance problem, especially in distributed storage systems. If you can get it under your control, you can definitely benefit from it.

I don't think there'll be much. Most modern relational databases rely on underlying filesystem to manage the persistence aspect. Many cannot even work with bare block device (I don't think PostgreSQL can). So, they need OS to provide at least that.

However, in principle, if you could design a database to run as unikernel you could benefit from creating your own persistence layer. For instance, you might be able to completely or to a large extent avoid the problem created by fsync.

Another aspect you could aim for is becoming a real-time database, because you'd have control over memory allocation and thread context switching. This may not give any tangible benefits to an average database user, but in cases where being real-time is relevant (eg. medical equipment) you'd certainly be able to expand your area of application.

So Nanos can run quite a few databases today and you should get the same sort of experience that running a database on any vm will produce (that is everything running in the cloud).

You are absolutely correct though that most filesystems in use today were designed to utilize and more appropriately put deal with various aspects of running on actual real hardware which is interesting as compared to running on a vm. I think there is a ton of room for newer filesystems to emerge that are tuned for virtualized workloads vs tuning them for hardware.

From my POV security and this particular implementation uses threads so automatically gets bonus points imo when dealing with the problems associated and discussed on the linked mailing list thread wrt large memory.

However, from a much more arguably important POV - usability - if you look at the vast collection of docker-compose.yml files out there a ton that want postgres to run. Yes, you could run it on the side and run the rest as unikernels but that breaks the UX of the compose functionality. It is such a small thing but having it goes an insanely long way towards having better developer experience.

We now have compose like support too, so you can take some app that spins up 10+ instances locally and along with the others and spin up postgres inside as well. So usability/dx.

Reply on news.ycombinator.com