crawshaw

Author here.

I need to fix our transfer pricing. (In fact I'm going to go look at it now.) I set that number when we launched in December, and we were still considering building on top of AWS, so we put a conservative limit based on what wouldn't break the bank on AWS. Now that we are doing our own thing, we can be far more reasonable.

Author here.

Almost every VC rejected us when we went to get seed funding for Tailscale, we knew none of them. Friends of friends of acquaintances got us meetings. Fundraising is very possible for you if you are committed to building a business. Most important thing is don't think of fundraising as the goal, it is just a tool for building a business. (And some businesses don't need VC funding to work. Some do.)

The biggest challenge is personal: do you want to build a business or do you want to work with cool tech? Sometimes those goals are aligned, but usually they are not. Threading the needle and doing both is difficult, and you always have to prioritize the business because you have to make payroll.

Author here. Most of our infra is custom, the VMM is based on cloud-hypervisor (a project spiritually similar to Firecracker). We have a lot of work to do, including on the VMM, but right now there is more value for users if we spend our time on the VM management layer and GLB.

Hi, co-founder here. If you would like more detail, and often HN does, I wrote up some more about what we are doing here: https://crawshaw.io/blog/building-a-cloud

This is an S-1 for Firefly Aerospace Inc.

Based on this and recent product releases, Anthropic seems keen on building a closed ecosystem around their excellent model. That is their business choice, I suspect it will work well. But I cannot say I am particularly excited to have my entire development stack owned by one company.

You can get this effect today by installing Tailscale on your exe.dev VM. :)

The reason we put so much effort into exposing these publicly is for sharing with a heterogeneous team without imposing a client agent requirement. The web interface should be easy to make public, easy to share with friends with a Google Docs-style link, and ssh should be easy to share with teammates.

That said, nothing wrong with installing tunneling software on the VM, I do it!

Amazingly even most p2p works with NAT, see (and I am biased here) Tailscale.

I certainly wish we simply had more addresses. But v4 works.

(exe.dev co-founder here)

We are not running out of IPv4 space because NAT works. The price of IPv4 addresses has been dropping for the last year.

I know this because I just bought another /22 for exe.dev for the exact thing described in this blog post: to get our business customers another 1012 VMs.

(exe.dev co-founder here)

IPv6 does not work on the only ISP in my neighborhood that provides gigabit links. I will not build a product I cannot use.

Even when IPv6 is rolled out, it is only tested for consumer links by Happy Eyeballs. Links between DCs are entirely IPv4 even when dual stacked. We just discovered 20 of our machines in an LAX DC have broken IPv6 (because we tried to use Tailscale to move data to them, which defaults to happy eyeballs). Apparently the upstream switch configuration has been broken for months for hundreds of machines and we are the first to notice.

I am a big believer in: first make it work. On the internet today, you first make it work with IPv4. Then you have the luxury of playing with IPv6.

That is very true. We use copy on write for exe.dev base images right now, and are accumulating a lot of storage because of version drift.

We believe the fix here is to mount the base image as a read-only block device, then mount a read-write block device overlay. We have not rolled it out yet because there are some edge cases we are working through, and we convinced ourselves we could rework images after the fact onto a base image.

Right now our big win from copy-on-write is cloning VMs. You can `ssh exe.dev cp curvm newvm` in about a second to split your computer into a new one. It enables a lot of great workflows.

Nice to see this work! I experimented with this for exe.dev before we launched. The VM itself worked really well, but there was a lot of setup to get the networking functioning. And in the end, our target are use cases that don't mind a ~1-second startup time, which meant doing a clean systemd start each time was easier.

That said, I have seen several use cases where people want a VM for something minimal, like a python interpreter, and this is absolutely the sort of approach they should be using. Lot of promise here, excited to see how far you can push it!

Send me an email!

exe.dev | Full time | SF Bay Area | multiple roles

Support Engineer - If you want to use Claude or Codex (or Shelley!) to trawl through our code base, augmented with (carefully scoped) API keys to make our customers lives better, we are hiring.

Designer - If you want to use Claude or Codex (or Shelley!) to make our product functional and beautiful, we are hiring. You do not ship mocks or assets to other teams. There is no other team. You ship by pushing to production. Don't worry, we've got your back.

Software Engineer - If you see a pattern here, we are hiring. Expect to design, build, and run entire subsystems. What matters is attention to product detail and overall architecture, we have agents for writing code.

With significant industry experience, pay will be over $200,000 with meaningful equity.

We are a small team and going to stay one. The focus is building a high-trust environment. Success for us is if you say "I'm going to fix the load balancer" we all sigh with relief, because you are on it and we can rely on you to take on and solve large projects.

The concern is not losing access to some new IDE for operating outside the terms of service. The concern is when you lose access to the IDE, you also lose access to your 20 year old Gmail account.

A general problem for Google products is that everything is mixed together.

But you also want smart phones, electric cars, and a navy. There needs to be a path towards doing things other than foisting them on people who are out of sight.

Lot of things could be added to this list. Good luck getting permission to start a hospital, or permission to mine/refine anything with a slightly messy process (e.g. rare earth metals). You can't build a new port. The California Coastal Commission won't let you open a new hotel anywhere on the water. You can't even keep a bar open late in San Francisco.

I do think sandboxes as a concept are oversold for agents. Yes we need VMs, a lot more VMs than ever before for all the new software. But the fundamental challenge of writing interesting software with agents is we have to grant them access to sensitive data and APIs. This lets them do damage. This is not something with a simple solution that can be written in code.

That said, we (exe.dev) have a couple more things planned on the VM side that we think agents need that no cloud provider is currently providing. Just don't call it a sandbox.

It is a good theory, but does it hold up in practice? I was able to prototype and thus argue for and justify building exe.dev with a lot of help from agents. Without agents helping me prove out ideas I would be doing far more boring work.

Hi, author here. I mean the piece of code that calls the model and executes the tool calls. My colleague Philip calls it “9 lines of code”: https://sketch.dev/blog/agent-loop

We have built two of them now, and clearly the state of the art here can be improved. But it is hard to push too much on this while the models keep improving.

You in SF? My schedule is a bit busy since we launched but I could find an hour in the city.

Glad to see this already expressed here because I wholly agree. Programming has not brought me this much joy in decades. What a wonderful time to be alive.

[exe.dev co-founder here] That's true today but I am rapidly buying IP addresses so we can increase that number.

Hi, author here! Our first system used usernames:

  ssh servername@exe.dev

We did not like stealing users for this.

I built jumpbox support into our proxy on a branch to get -J working. I have not shipped it (but probably will in the future). The reason we did not use it for default access is we do not want our proxies to complicate the experience of using VMs. We want to be as close to “just a computer” as possible. Jumpboxes are not unheard of but they are still a relatively uncommon ssh feature.

It gets even nicer if you setup a CNAME for your box, then you can:

  ssh myserver.com

...and get right into your exe.dev VM.

Another way of saying it: we are willing to take on a lot of complexity internally to remove a little bit of complexity for our users. This was my standard of operation at Tailscale and it is how I like to program.

That is very good framing for the problem with advertising today.

It also says that if you can find an audience to build a new ad platform, the incumbents have created enormous margin for the new platform.

Years of debugging in that company’s restricted environments solidified my desire for shell access to production environments. I was there a month before I was hunting for breadcrumbs in a BINARY_INFO log that I had five minutes to grab before it was deleted.

Mine (prometheus) doesn’t because there are a lot of high-dimensional values to track in /proc and /sys that would blow out storage on a time-series database. Even if they did though, they could not let me actively inject changes to a cgroup. What do you suggest I try that does?

Happy to try an alternative. Traces I have tried, and it is not an alternative.

That is a command line tool run over ssh. If you have invented a new way to run command line tools, that’s great (and very possible, writing a service that can fork+exec and map stdio), but it is the equivalent to using ssh. You cannot run commands using traces.

How did you use tracing to check the current state of a machine’s iptables rules?