Isn't that the whole idea of unikernels?
And the answer (IMO) is that in the end your unikernels are just fat applications and your hypervisor is just an emaciated OS. The abstractions are at the wrong places and you can't properly optimize resource usage because the hypervisor has to treat the guest OS as a black box. Otherwise it's eventually going to cease being a hypervisor and be a cruft-laden kernel.
If our only concern was to get the most performance out of a single piece of hardware, that would be the end of the discussion. But, that's not the problem this kind of architecture is setting out to solve. We've already reached the point where raw performance on a single box is less important than being able to virtualize. EC2 proved that a decade ago; we're willing to trade raw performance for the flexibility of virtualized infrastructure in some cases. Hell, EC2 is about twice as expensive as similar physical hardware (even today); but subtracting the cost of managing the hardware and adding the ability to spin them up and down makes them more cost-effective for some deployments.
So, sure, people are trying to make it efficient. And virtualized and container based hardware has improved remarkably in that time (virtual disk and network I/O is now very close to bare hardware performance, whereas it was very far from it in the early days). But, even when they fail to achieve near peak performance there's still a compelling benefit for many uses.
The idea is to have each app in a "ready-to-run" state, so that it can be popped into existence anywhere on a pile of infrastructure and shutdown when it's not needed anymore. A container+app is a self-contained unit. If you're running several apps on an OS, and you update the OS, you now have to test all of the apps to be sure the update doesn't break things. If a security vulnerability happens for any of those services, you may no longer be able to trust the entire system. Apps that misbehave might impact all of the other services on the system. Containers are not a panacea, mind you...they introduce new headaches of their own, and the way a lot of folks are building and deploying containers is a joke (it's been a great excuse for people to start pushing out big binary blobs of crap again, which was a thing I'd hoped was behind us years ago).
But, the principle benefit is that a container provides a level of abstraction at the service level rather than the application level. This "thing" (which is a container that has an app in it) can be spun up anywhere and do its job. It can be upgraded or modified (including libraries and other packages) without impacting any other thing in your stack. Part of a self-healing infrastructure is being able to think of any service as a cog in a greater machine, rather than some individual piece of complexity you have to interact with to keep it running. And, ideally, we'll be able to outsource maintenance of those cogs to someone else, the way we outsource packaging binaries to OS vendors, today. Right now, that's a mess, and most of the people building containers have no clue what they're doing; so it's a minefield. But, it won't always be so.
It's just another abstraction for us to work with, is what I'm trying to say. A container can be a black box that we only need to open up occasionally (or never, if we trust our vendor to build it right).
"If it is something better, then why don't we write applications to be that way to begin with? Serious question."
We probably will, eventually. At least, it'll be closer to that than big full OS + a bunch of apps running on it.
The benefits are many; the costs and tooling are still high and fragile. But, it'll get better, and we'll get more fine-grained in how we're deploying these pieces. Most people, including myself, still think in terms of "I have a server, now I put things on it, and they combine to make a website/mail server/whatever". That's probably the wrong way to approach big web scale problems (though the number of people actually managing web scale systems vs. server-scale systems is probably small; most web sites and mail servers will never need to expand beyond a single server, and it's probably a stupid waste of time and resources to do otherwise).
The future of big web service looks like Kubernetes or something along those lines (one-app per container, very small and focused OS images, somewhat ephemeral in nature...they will come and go). It does not look like one big server with a bunch of apps running on it.
Containers don't actually contain very well on Linux (yet, though some nice things are happening on that front, and you can already build very solid things with SELinux, but I haven't seen anybody actually doing so; except probably the Project Atomic folks).
But, yes, in theory, a container-based deployment is hella tight on the security front. One service being compromised is no big deal: Kill it and instantly spin up a new one on a new IP (hopefully with a fix for the security issue). If your visibility into the system is high enough, you may even be able to architect detection of compromises; e.g. if your container for database service starts sending email or opens an IRC port, you know it's fucked, and you kill it with prejudice. In a system built to think of containers as black boxes with APIs, you don't need to keep any particular one around if you have reason to tear it down.
Note that I haven't actually seen people doing any of this with containers. But, it's a thing that one could do that cannot be done when all of your services are applications running on a single big machine. The narrower the purpose of your container, the more secure it can become. You still have lots of moving parts in your total infrastructure (more, actually, since the container management has a cost, too), but each moving part has very well-defined boundaries, and misbehavior is easier to spot and easier to shut down quickly and via automated means.
Operating system upgrades.
Configuration management.
Startup/teardown of multiple instances.
Symbiotic or dependent processes/programs.
Programs not specifically built with isolation in mind.
Inter-application security.
For example, running a tiny jobs server:
* Scheduler in one process which spawns
* Application code running in many subprocesses
The scheduler might be in Ruby and the jobs might be in Ruby; or the scheduler might be Cron and the jobs in shell. In either case, if a job crashes the other jobs and the scheduler are very likely to carry on their work.
It is also nice that you can bring together tools from different communities into a single application. A "Rails" container might utilize Nginx (C), Rails/Unicorn (Ruby) and Node (JS/C++) together. A single statically linked application would, barring some really amazing innovations in something like Clang modules, have to be in one language therefore from one community.
Erlang solved a lot of problems of scale in very effective ways. These new architectures are solving the same problems in ways that look similar from a distance. This new(ish) approach just means we can use existing software mostly unchanged, while still getting a lot of the resiliency benefits of something like Erlang; doing it with Erlang means everything has to be written in Erlang. That wouldn't be a crazy idea for some deployments. But, for others, it's not tenable.
Not necessarily. If they all can compile to native machine code with the same C-like semantics, they can (in theory) be combined in that sort of manner. This is true of both Rust and Go, last I checked, as it is with pretty much any compiled language that has a "convert to C" step (Chicken Scheme is an example that I'm particularly familiar with) or perhaps even a "convert to LLVM IR" step.
Not that this is especially feasible for the majority of languages being used in web development (at least by those with a lick of sanity; Go and Scheme are the only exceptions that I'm aware of off the top of my head), but it's still worth considering.
I'm sure the Plan9 ABI variant used by Go makes more sense than the platform ABI, but supporting the platform ABI for FFI doesn't add much complexity at all to the compiler.
I'm 91% sure my point still stands with Rust, though, at the very least if everything's compiled through LLVM (and uses the same calling conventions).