aliguori

Restricting /dev/kvm these days doesn't make much sense. The interface is designed to be safe for any user. The fact that we started as a character device and not syscalls is just a historical decision.

I love QEMU, it's an amazing project, but it does a ton and it's very oriented towards running disk images and full operating systems. We wanted to explore something really focused on serverless. So far, I'm really happy with the results and I hope others find it interesting too.

There's a lot to love about NVMe and timeouts are not actually part of the NVMe specification itself but rather a Linux driver construct. Unfortunately, early versions of the driver used an unsigned char for the timeout value and also have a pretty short timeout for network-based storage.

As mentioned elsewhere in the thread, recent AMIs are configured to avoid this problem out of the box.

As with all virtual and physical machines, patches are necessary to protect against process-to-process Meltdown within the OS itself. Those are starting to roll out from the respective vendors although it will take time for those to work inside a PV instance.

Both i3.metal and c5 use the same underlying Nitro technology.

Disclaimer: I work at AWS on the team responsible for the Nitro System including EC2 Bare Metal Instances.

We will have some more details on how this all works at re:Invent in a couple weeks.

That word doesn't mean what you think it means :-)

If you break into QEMU, you should be a non-privileged user. If you are using libvirt, you are in a cgroup based jail (basically a container) with SELinux being enforced too. So after breaking into QEMU, you would still need to break out of the container before you could attack anything.

But Nelson's exploit was pretty cool. I initially thought remote code execution wasn't possible and he turned around pretty quickly with the exploit. It's quite impressive.

However, I have found that a lot of the benefits of virtualization are often lost because of the flexibility of having a full blown Linux OS. People stick too many services on the host and/or don't know how to properly allocate resources.

I think there is something to a simplified model where you no longer have access to any host environment and can only "fill slots". In practical terms, it makes it easier to implement features, like firmware-style upgrade and rollback of the host, for the masses.

Yes, you can do this with a full blown Linux OS on the host but it requires an operational discipline that most environments don't seem to possess.

I'm not sure I buy the whole "run multiple containers on top of an EC2 instance" thing though. I understand why they're using LXC vs traditional virtualization but it seems to me like it's a case of solving all problems by adding another layer of abstraction.

There are too many different kinds of virtualization so we'll just add another layer of virtualization.

The problem is that the PRNG has a weak default entropy source. The same problem existed in the kernel for ages. See http://www.factorable.net

The real advice here ought to be that if you are building an application that generates keys, you should make sure that the system has appropriate entropy before generating the keys. Don't assume the PRNG (whether it's /dev/urandom or OpenSSL) does it for you.

Steal time exists to fix a problem. When a hypervisor needs to pre-empt a running guest, without steal time, when the hypervisor eventually resumes that guest, as far as the guest can tell, the process that was running when the whole guest was pre-empted had run the entire time.

This means that if a guest is pre-empted, then CPU usage reporting in the guest becomes horribly wrong with some processes having much higher reported usage than they actually got. This affects fairness and can cause lots of bad things.

Steal time is simply a way to tell a guest that it was pre-empted. The guest OS can then use that information to correct its usage information and preserve fairness.

However, it is not a general indication of overcommit. When a guest idles a VCPU, that VCPU will be put on the scheduler queue. It may receive an event that would normally cause it to awaken the VCPU however if the system is overcommitted, it may take much longer for the VCPU to be woken up.

Most clouds are designed to allow multiple VCPUs per physical CPU too and there certainly is capping in place. You can still see steal time even though you are getting your full share.

Let me give an example:

1) You are capped at 50%. You run for your full 50%, go idle, the hypervisor realizes you've exhausted your slice, and doesn't schedule you until the next slice. No steal time is reported.

2) You are capped at 50%. You have a neighbor attempting to use his full time slice. Instead of getting to run for the first half of your slice with the neighbor running for the second half, the hypervisor carves up the slice into 10 slots and schedules you both in alternative slots. Both guests see 50% steal time.

You will get the same performance in both scenarios even though the steal time is reported differently.

I am intimately familiar with eWLM and I think it's quite unfair to call it containers.

> It's your turn to provide some citations. You could argue that the Linux kernel has failed to achieve good multi-tenancy support, but saying it hasn't been designed for multi-tenancy just makes you look ignorant of the last five years of development on namespaces, cgroups, and general multiprocess resource scheduling.

I'll give you a simple example. Do dd if=/dev/sda of=foo.img (or even /dev/null) in one container and measure I/O performance in the other.

The page cache is a global resource and so far there is not way to isolate it within containers. Buffered I/O is pretty fundamental to all workloads.

>> I'm quite sure it's not faster than virtualization. > > Citation needed, especially since you're arguing that more layers of abstraction (kernel + hypervisor + kernel) is at or above the same speed of fewer (kernel + cgroups/namespaces).

cgroups aren't free. See https://www.berrange.com/posts/2013/05/13/a-new-configurable...

I already cited a widely published benchmark (SpecVIRT). You article cites made up numbers (5-10 minutes to provision a guest).

$ qemu-img create -f qcow2 -b template.img new-guest.img $ qemu -hda new-guest.img -enable-kvm

And I have a nearly instant guest under KVM. So I don't know where your 5-10 minute number comes from but it's clearly not correct.

Note that I'm not normally one to bad talk anything. Containers are fine for that sort of thing. But you're claiming that containers obsolete virtualization and that's just plain silly.

I expect better from you :-)

> Mature containers have been around since the the days of mainframes.

Citation needed.

I'd go as far as to say that there is no such thing as a mature container technology. The fundamental problem of containers is that mainstreams kernels are not designed to be multi-tenant. They support multiple users quite well but when you try to have multiple root users, badness ensues.

Even the best container systems today for Linux still have fundamental gaps.

> Even if hypervisors achieve CPU performance parity for running systems, you still haven't addressed the memory and storage overhead (in terms of resident footprint) of running full OS images instead of containers.

Re: memory, same page merging can eliminate a lot of that overhead when running mostly homogenous workloads. Re: storage using CoW not only makes provisioning instant (not 5-10 minutes) but also addresses the storage concern.

If you mean the overhead of running two kernels, well, actually using namespace has a fair amount of practical overhead too.

Of course, benchmarks speak louder than words here. No one has published a container based result for SPECvirt because I'm quite sure it's not faster than virtualization.

You can be compelled to testify under oath for many reasons none of which involve doing anything wrong. When you testify under oath, you must answer the questions asked truthfully or face unlimited jail-time for contempt.

The fifth amendment is the only form of protection you have from being called to testify and being grilled about any random topic. It's a balance to the otherwise tremendous power that a court has.

The prosecution must be able to make their case without the defendants assistance. This is a fundamental aspect of our legal system.

Has more details. Looks like it's KVM.

Any crude comment relating to sexuality would fall into this category. I am not attempting to justify the response here, just explaining the rationale.

KVM is able to achieve this level of scalability because it leverages Linux which is already very good at scaling to large systems. I wouldn't expect this kind of scalability from other hypervisors any time soon.

If you own a business headquarted in MA, aren't you earning income (through the business) in MA?

If you work in a different state (even if you don't live in the state), you still owe income tax for income earned in that state. This is non-resident state income tax.

Pretty sure MA has a reasonable claim here based at least on what the author states.

Both KVM and Xen use QEMU for their device models so the machine they emulate is very similar (at least in HVM mode). Xen and KVM use different paravirtual I/O drivers but everything I've ever seen indicates that I/O performance is better in KVM.

Xen PV guests are a mixed bag. On the one hand, they work on boxes without virtualization hardware (which are rare anymore) but on pre-Nehalem/Barcelona processors, they are much slower than using HVM (especially 64-bit).

I've never seen a good Xen HVM benchmark either comparatively speaking... Of course, I'm more than a little biased :-)

Besides not being able to distinguish between a true binary blob and a string, the two parties need to agree on what string encoding should be used. Is it UTF-8, UCS-2, EBCDIC? I suspect this would create tons of incompatibilities between implementations as various parties make their own naive assumptions about what strings are encoded with.

That seems like a pretty major flaw to me. X.690 is a bit overly complicated (there are 10+ string types...) but there is such a thing as too simple.