CompuIves

That's so great to hear, thank you!

It might be interesting to try out the CodeSandbox app: https://apps.apple.com/nl/app/codesandbox/id1423330822?l=e.

It's the same concept, you get a VM you can work on from a native app, but our app is for iOS only and supports iPads natively.

Disclaimer: I'm one of the co-founders of CodeSandbox.

Thank you for creating this! I remember using this for the header of my portfolio website before I got my first job. Happy to see that it's still active, I hope that the monetization model is working out well!

That would work! You could even run the Postgres inside the VM, and the data inside the DB will be cloned as well between clones.

Yep, that's exactly it. So the moment the fork happens, we create both a new layer for VM A and VM B, so they can both use the same base layer.

That is very interesting, would it also be able to handle paused VMs where it buffers the packets up to certain threshold?

Thanks a lot, I appreciate that!

Right, that makes sense. Once the memory page is in memory it should be fast though. We use shared mapping right now, and practically the pages stay in memory during the lifetime of a VM once they've been loaded, but we need to do more testing when there's more memory pressure.

I've been looking at huge pages recently, I'm going to do some more testing with transparent huge pages today and see if it changes performance. Unfortunately we cannot use reserved huge pages because that doesn't work with shared mmap on say an XFS FS.

Another idea is to make clones use the same memory base layer of their parent, then the pages are already prefaulted and it would deduplicate overall memory usage. Many things to discover still..

You're completely right. Right now we're hosting the VMs on bare-metal at Hetzner, and we're looking at OVH or Contabo for hosting in the US.

So let's say VM A is already running, and it's cloned to VM B. When that happens, we freeze the memory of VM A, and link it to VM B. For both VM A and VM B, any new write will be done to a new layer.

The TCP connection gets "paused", it doesn't get broken but packets don't arrive. The packets that don't arrive are seen as packet loss, and so they get resent. If the connection stays frozen too long it will lead to disconnection (at least of the websocket connection to the VM).

For IP uniqueness, we give every VM the same IP, but we put every VM in its own network namespace. Then we have iptable rules to rewrite the src/dest IP on every packet that enters the network namespace.

Yes, and this is also the biggest challenge. Right now we use XFS to enable CoW, but that quickly leads to filesystem fragmentation. I'm still looking at a way that we can quickly let both VM A and VM B use the same base snapshot, and write new changes to either anonymous memory or a file.

This is something I will look into! I'm thinking it could reduce start time because we have to copy the mem snap from disk to tmpfs, essentially loading it into memory, but I'm going to try this!

Yes! We're watching cloud-hypervisor as well, it might also be more suitable for our use cases.

Yes, we did look at live migrations since there's a lot written about it and it's the closest to cloning a running VM. Lots of development in that space!

Exactly! We're looking at the second use case, to have a VM running tied to the git history. Right now we do this on a branch level, so you could say "connect to branch X on microservice Y" from another microservice, and you can test APIs quickly. There's a lot of new things this enables!

Yes it's a good point, this is one of the reasons that we wanted the fork time to be low. If we keep it low enough, the connection won't break and other users won't notice it. That said, for some things (like terminal access) it's impossible to hide it.

Practically, 99% of the forks will be done from the `main`/`master` branch of the repo, which is read-only for everyone on the team. So the mini-pause isn't breaking in those cases.

I'm very eager to see more developments in the fresh start times!

The main reason why snapshotting became interesting for us, is because we're running development servers defined by our users. A development server could take a long time to start, sometimes minutes.

So even if we can start the VM fast, the most important speedup for us is on the user code that we cannot control.

Could you share why you're seeing such a slowdown? I've mainly experienced a slowdown when loading pages for the first time, since new pages need to be page faulted into memory from disk first, but after those pages have been loaded into memory I don't experience any slowdown compared to fresh starts. That said, that's only based on how I've experienced it with the type of workloads that we run on the VMs.

I don't believe Firecracker currently supports GPU (latest what I saw about it is here: https://github.com/firecracker-microvm/firecracker/issues/11...). But I wouldn't be surprised if there's another MicroVM manager that would support GPU + snapshotting.

It could be shared memory all the way through, but the memory of the original VM should become read-only once a cloned VM starts reading from it. So then both VMs (the original VM and the new VM) should put their writes in a new CoW layer.

Using XFS with CoW has been the easiest way to enable this, but if there's a way that we can do this purely in-memory, that would be even faster.

That said, for hibernation we would still have to persist to disk, but timing is less important there.

Thank you! Right now our main use case is cloning development environments so we can provide a fresh running dev env for every branch and PR. However there are many other interesting applications, like speeding up CI jobs with VMs that start from a snapshot.

I'll make sure we write about the other topics as well. For the network, we run the VM in its own network namespace on the host, and we give every VM the same IP. We then use an iptable rule to rewrite every incoming and outgoing packet to the IP that the host has assigned for the VM.

The biggest VM we've been running for dev environments have 12GB RAM, 8vCPUs and 30GB disk. I've also done some tests with 16GB RAM and that worked well too. Have yet to find an upper limit.

Another (unrelated) test we've done is on overprovisioning memory. We were able to run 200 VMs (all running Vite dev server where a file was changed every second) with 2GB RAM per VM, on a node with 128GB RAM. Because we were mapping the memory files on disk directly to the VM, the VM would automatically "swap" the memory back to the memory file when it had memory pressure. The bottleneck here was CPU.

Firecracker keeps a bitmap of which pages have been dirtied (it's a flag you can turn on), so you can make incremental snapshots of only the changed pages (more here: https://github.com/firecracker-microvm/firecracker/blob/main...).

In our case we changed Firecracker to use a shared mmap instead of an private mmap, so in our case the dirtied pages were synced back automatically to the backing memory file. The main reason for this was to reduce IO on snapshot time. I'm also looking at other ways we can do this, because using a shared mmap fragments the underlying xfs fs pretty fast. Maybe we can batch writes more instead of writing single pages.

Thanks for sharing the paper, that is super interesting. This is indeed a challenge. The fact that we run development workloads instead of production workloads makes this easier for our use case. We do some rehydration, and when forking across team/organization boundaries we don't clone if there are secrets set. But for production workloads this would not be ready yet.

Once uniqueness has been solved though, VM cloning would become a real solution for serverless hosting (and many of other cases), exciting prospect!

Firecracker is one of the most exciting technologies I've seen in a while. It felt like a whole world opened up after learning about its capabilities.

Send me an email at my name @ codesandbox.io, then we can plan something!

This looks amazing, really nicely done.

I'm the co-founder of CodeSandbox and it's super cool to see Sandpack used this way in the wild! We've been working on a way to run VMs with Sandpack, maybe we could work together on it if you're interested!

Ahhh I see. Yep, we should have better messaging there. With CodeSandbox you can run _anything_, since we allow you to run any Docker container. I've been running it with Elixir and Rust as well. We do have a focus on making the web development experience seamless first though.

The plans currently don't reflect the beta of Projects, there we do have specs (by default 4vCPUs + 4GB RAM) with upgrades up to 12GB RAM and 8vCPUs. We're still exploring other machine specs there too.

The place where this differentiates is that every branch runs its own dev env, based on a memory snapshot from the main branch. This way when you switch branches, you switch to a new environment and you won't have to think about running migrations/installing dependencies/pulling changes just for that branch.

What would you say makes it more limited? In the end you get a VM where you can run Docker/Nix etc...

Yes, after every compilation we send the full bundler state. This is saved on the context so you can use that to analyze the output. In one of the examples in the blog post we show the transpiled code for example.

A hidden "feature" is that we automatically run tests defined (e.g. index.test.js) using jest, and dispatch the test results. So you can listen to that as well.