Comment by colinchartier | Hacker News Reader

Comment by colinchartier | Hacker News Reader

Hey HN,

We're Lyn & Colin and we built Layer - It creates a unique staging server for every commit.

Before Layer, I was CTO of a startup with a 10-person developer team, and dealing with staging servers (and end-to-end tests) was one of the most annoying parts of our workflow.

Layer lets you define staging servers the same way you define Docker containers, and get a unique one for every commit going back a week. We run them in the cloud, so there's no need to pay for AWS servers you're not using.

Also, if you're running a webserver, you get a unique URL for each unique server automatically so that you can post it to a slack channel and get feedback immediately.

When we're setting up the staging server, we use the same sort of cache as Docker so that you can skip repetitive tasks like setting up a database or running database migrations if they haven't changed.

Would love to get your feedback, it only takes five minutes to do the onboarding tutorial.

How would this handle external state? I realize in general that's an impossible task, but specifically I'm thinking of something like a database. Is there a way for each "server" to get a private copy of the database? Otherwise it seems like if I create a non-backwards-compatible migration (let's say I drop a table that's no longer necessary) I could break servers that were spawned for other commits.

Glad you asked! If you do the Django tutorial after clicking "try now", you can see an example of a postgres database that gets set up for every run.

Since we give full VMs instead of just containers, you can keep the database running in the background. Because we do memory snapshotting, you could turn a 5 minute database start + migrate + populate into a 1 second "restore from memory snapshot"

> Since we give full VMs instead of just containers, you can keep the database running in the background.

Interesting contrast to flockport that runs multiple processes in a LXC [0] (treats it like a lightweight VM).

[0] https://news.ycombinator.com/item?id=17478736

Hey HN!

I'm Lyn, co-founder of Layer. Here's a quick 5 min demo video in case you don't want to do the sign up tutorial (even though it takes 5 mins toooo): https://www.youtube.com/watch?v=PH-n70gPQgA

Happy to answer any questions here.

How is this even possible for $35?

50 staging servers per seat ($1450 Value) 11GB RAM per server 8 CPUs per server

We spent six months developing a hypervisor that lets us treat any linux VM as a lambda - We pass on the efficiency to our users as a value add.

You should probably be selling that if it works well. That is something I have not seen.

<mostly sarcastic>

Bakevm with tooling > create userdata script for cloud init to copy payload to server > create systemd oneshot service to run payload && shutdown

You should try it! Our baking interface looks more like Docker than a traditional VM provision (ala Packer)

Also, we keep the baked images around for a week and trigger them whenever they are requested (with, say, HTTP, or terminal access)

I may be misunderstanding the product, but it seems to be able to run a webserver on each "server".

More info on that would be cool. Something like Amazon's FireCracker?

Sort of - FireCracker is less suited for testing use-cases because it still cold-boots. We use the same kernel features that let laptops resume when the lid is opened instead, they're less mature for production but much faster for staging and testing.

Ah. VMware does something similar for hot provisioning VMs from a point in time memory snapshot.

So you guys boot the image in the background, then capture a memory dump of it so you can quickly launch VMs from that snapshot? Or you boot the VMs the traditional way the first time, and then just suspend them when not in use?

We do even more, we take snapshots as the "Layerfile" runs - this basically lets you merge the memory dump semantics you mention with the interface of Docker.

Also, we monitor which files are read by any step in the process, and let you skip setup chunks that don't need to run (i.e., installing libraries) without needing to micromanage what files you copy into the staging server.

Man that sounds cool. I don't have any use for something like that, but I would love to see it opensourced to play with it (which I assume you guys won't be doing anytime soon)

Are you using socket activation or something similar? I think there are really interesting use cases around that paradigm for packing services onto one host.

Some blog posts I remember from when I was exploring this idea a few years ago:

http://0pointer.de/blog/projects/socket-activated-containers...

https://www.ctl.io/developers/blog/post/running-drupal-in-li...

Sort of, you can think of it as an entire socket-activated OS.

Our hypervisor does the logic though, we don't rely on the OS of the staging server itself for anything (think PXE)

Why are you trying to tackle some small little market of staging servers for the handful of companies that actually care about this?

Take your Hypervisor and sell it to companies that want to efficiently lift-and-shift legacy systems in to the cloud and make your investors super happy.

First of all, this looks awesome and go Canada! Random comments:

Is there a way to try without giving access to my github account?

Less importantly: Can you put your email in your HN profile? Can you add LayerCI Twitter handle to your site?

1. Not yet unfortunately, we didn't want to waste time creating a separate auth system when we only integrate with GitHub for now anyways. To see a short demo though, here's an earlier one we made: https://www.youtube.com/watch?v=PH-n70gPQgA

2. My email (for Layer) is colin@layerci.com

3. Done!

Thanks for your feedback.

How do you handle very complex and integrated architectures like a monorepo with many microservices?

How do you run your layer file locally?

Can you describe automated tests in a layer file?

I built something similar[0] to this for integration testing and those were my biggest hurdles. At a previous job where all of the code was python/node/java I was also able to generate cross-service end to end code coverage from these tests and that feature was extremely helpful as it helps you remove redundant tests.

Your service looks very polished and I'm extremely impressed someone is taking this idea to market.

0- bunt.build

> monorepo with many microservices

We scan the entire repository for Layerfiles, so you could, say, create a layerfile per service for unit tests, and a bigger Layerfile in the repo root that does the whole deployment and runs a few E2E tests.

> How do you run your layer file locally? They can be approximated by running a docker container and just copy/pasting the commands. We haven't built this yet because I know it will be a huge time sink to make it reliable (see, e.g., vagrant)

> Can you describe automated tests in a layer file? We tell you if any step fails (e.g., returns non-zero exit code) so you can just as easily run "./deploy.sh" as "./test.sh" - you can also start a webserver and run E2E or acceptance tests against it, for example.

> generate cross-service end to end code coverage We are trying to avoid anything that isn't very general - it's possible to run https://codecov.io/ within Layer if you want functionality like this.

Reply on news.ycombinator.com