574 karma · joined September 6, 2017
Blog at https://blog.kylegalbraith.com
Previously at: Era Software, Thorn, Zapproved
Previously built: https://ultimatefantasysupercross.com/ https://kylegalbraith.com/learn-aws/ https://kylegalbraith.com/learn-by-doing/
kyle@depot.dev
All of that to say, I think the t3/t4 instance used in this test is a bit problematic for getting a true idea of performance.
[0] - https://depot.dev/
But, I do think it helps awareness as we do get some folks that tell us I saw you on this repo or that.
We also do it to support projects we think are neat/useful to others even if we don’t use it.
I think there are some logic things to get right at the API level, like should I use events or contact properties to trigger loops? We're working on some of that and wish the guidance was a bit better/clearer. At the moment, any properties you send with an event get added to the contact, so it seems like contact properties are the way to go.
My last request would be to support array properties on contacts, as a given contact could be in multiple "things".
But in terms of a written piece of technical content, this is brilliantly written. Easy to follow and stay engaged. Well done.
The visibility into what emails have been sent, to who, and what the content was is also incredibly helpful when we are talking about transactional emails. Double bonus for being able to share that email as well.
In that deployment model, the Depot control plane passes changes to make in the customers environment via a small agent they run in their account. Here is some docs we put together for anyone that wants to go into a bit more of the details: https://depot.dev/docs/self-hosted/architecture
We should definitely be able to help here.
For now, if you want to sign up for our free tier we can flip your org to this type of structure you are talking about, 1 project, 1 user, first 60 build minutes free & 5 cents/minute after that.
But there is a few steps to get there in that setup. I believe you have to have three CodeBuild projects, one for each architecture & then the manifest merge. So it works but is a bit of config to stitch together.
With Depot, you would just install our `depot` CLI in your config and run `depot build --platform linux/amd64,linux/arm64` instead of `docker build`. We handle building the image on both architectures in parallel and can push the merged result to your registry. We can even run the builders in your own AWS account so you maintain total control of the underlying infra building your image.
We are working on other features for Depot that would go beyond the speed & multi-platform capabilities. We want to surface insights and telemetry about image builds that could help them be smaller, faster, and smarter. We are also thinking about things in the container security space such as container signing, sboms, etc. Happy to answer more questions about any of this!
Couldn't agree more with the pain points you mention, it was the biggest things that led us to start Depot and we're really excited about what we can do next.
If you ever want to do way with your hacky build process and try out Depot, we have a free tier now.
Here is my direct email if anyone wants to chat/learn/share more about this problem space, kyle [at] depot.dev.
Behind the scenes we are running the same engine as Docker, BuildKit, on VMs in AWS for both architectures. We combine native architecture support with a 50GB persistent SSD cache that is automatically available across builds, so no more saving/loading cache over the network.
All those things combined makes multi-platform or Arm image builds in GitHub Actions ~15x faster using Depot (example: https://depot.dev/benchmark/temporal). And it's a one line change, swap `docker build` for `depot build` or use our `depot/build-push-action` instead of `docker/build-push-action`.
We don't limit concurrency on the builder VMs. When you create a project you are effectively getting a dedicated build VM with a persistent SSD cache for each architecture you want to build for. We run native BuildKit on the VM and connect your `depot build` directly to the builder instance via mTLS. So your concurrency is unlimited with Depot as well (or at least as unlimited as BuildKit will allow for).
So your pipeline would effectively look like this.
image: node:10.15.0
pipelines:
default:
- step:
script:
- curl https://depot.dev/install-cli.sh | DEPOT_INSTALL_DIR=/usr/local/bin sh
- npm install
- npm test
- docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD
- depot build -t <my.container.registry.io>/<my_app>:<my_tag> . --push --project <DEPOT_PROJECT_ID> --token <DEPOT_PROJECT_TOKEN>
services:
- docker
Here I have used `--token` for passing in a project token to authenticate the build with your project in Depot. This can also be passed as an environment variable, `DEPOT_TOKEN`.Edit: Formatting. Also thank you for pointing out this is an integration we are missing in our docs. We will get that added!
I believe Cloud Build has no persistent caching so you are forced to use remote cache saving and loading. Which can incur a network latency that can slow the build to some extent. Cloud Build with Kaniko also expires the layer cache after 6 hours by default.
GitHub Actions is similar except that there is the ability to store Docker cache using GitHub's Cache API via the `cache-to=gha` and `cache-from=gha` directives. However, this has limitations like only being able to store a total cache size of 10GB per repository. You also have network latency for loading/saving that cache as well.
With Depot, the cache is kept on a persistent disk. So no need to save/load it or incur network latency doing so. It's there ready to be used by any builds that come in for the given projects.
The first thing that comes to mind is multi-platform support. Currently, it's very slow to build multi-platform images in things like Cloud Build because you have to emulate the Arm portion of the build. We don't have timings in Cloud Build for this, but in GitHub Actions a multi-platform image build can take over an hour when building for both Intel & Arm. With Depot, those builds can take less than 5 minutes because we build the image for both architectures in parallel on native CPU architectures.
Other things where Depot might be more streamlined than Cloud Build is that persistent caching is automatic. The dedicated build machines for each architecture have persistent SSD caches attached to them, so you don't need to save/load cache in between runs, it's just there ready to be used.
We currently charge on a per project basis as behind the scenes we are provisioning and maintaining a VM with a persistent SSD cache for each architecture you are building for (i.e. Intel & Arm). We switched to usage based pricing before rolling Depot out to the public so you are only charged for your active projects and used build minutes in a month. So far it has seemed to be working well, but we are always open to feedback and ideas on how we can make pricing better.
We don't currently document this, and we probably should, but it's entirely possible to use a single Depot project for multiple Docker images. There is no limitation that prevents this and the only downside is you may fill up your cache space at a faster rate and get fewer hits if you have different image builds clobbering the cache.