Comment by ncmncm | Hacker News Reader

Comment by ncmncm | Hacker News Reader

When I make a system where a unikernel could do the job, I usually want a full OS for program setup and initialization, and then an isolated core for the main loop, sharing (single-writer) memory pages with other, less performance-critical processes for logging, stats reporting, and any needed file system activities.

The makers of top-performing NICs have been quite good at providing direct user-space access to their hardware, typically by exposing a ring buffer in shared memory, and maybe mapping device registers too, so that the process on the isolated core never does another system call until shutdown weeks later.

It is some hassle to get customers to add boot flags (isolcpus=, nohz_full, rcu_nocbs=, rcu_nocb_poll, hugepages=, etc.), and to put any mapped files in /dev/shm or /dev/hugepages so the kernel won't invent excuses to block the procees, and to direct irqs to other cores; but unikernel setup is probably not simpler.

So, I'm not sure what a unikernel would get me. Portability, or independence from proprietary drivers?

I don't think the tuning you refer to is sufficient to bring the platform noise to levels required for some workloads. You will still see a lot of system call interrupts, TLB shootdowns, timer events, etc. Funnily enough yesterday I published an article tangential to this problem field.

http://bitcharmer.blogspot.com/2020/05/t_84.html

I'm not an expert on unikernels but my assumption is that you will see none of that OS jitter.

Good writeup, but is there any mitigation for TLS shootdowns?

TLB shootdowns are a product of multithreading, and of unmapping memory. Avoid one, the other, or both, and TLB shootdowns fade out of the picture.

You still have cores to isolate, busybody kernel threads to suppress, and hardware interrupts to direct elsewhere, but TLB shootdown paranoia is largely a product of the current fashion favoring multi-threading over running separate processes with carefully chosen sharing.

If you have only one thread running in the process, memory usage is static, any mapped pages are writable only by the one process and are not backed by external storage, you have turned off timer interrupts and rcu reaps, and do not perform system calls, where are these TLB shootdowns and timer interrupts coming from?

Recently I found out ebpf jitting will cause interprocessor interrupts even with all the isolation configs available today. There's probably many more places where such interrupts could be generated and not handled by the current isolation mechanism. Not saying a unikernel is directly a solution but CPU isolation isn't perfect either and could use something from a unikernel.

Reply on news.ycombinator.com