746 karma · joined September 5, 2017
Docker had a choice of markets to go after, the enterprise market was being dominated by the hyperscalers pushing their own Kubernetes offerings. So they pivoted to focus on the developer tooling market. This is a hard market to make work, particularly since developers are very famous for not paying for tooling, but they appear to making a profit.
With Docker Hub, it's always been a challenge to limit how much that costs to run. And with more stuff being thrown in larger images, I don't want to see that monthly bill. The limits they added hurt, but also made a lot of people realize they should have been running their own mirror on-prem, if not only to better handle an upstream outage when us-east-1 has a bad day.
Everything else has been pushing into each of the various popular development markets, from AI, to offloading builds to the cloud, to Hardened Images. They release things for free when they need to keep up with the competition, and charge when enterprises will pay for it.
They've shifted their focus a lot over the years. My fear would be if they stayed stagnant, trying to extract rents without pushing into new offerings. So I'm not worried they'll fail this year, just like I wasn't worried any of the previous years when similar posts were made.
There's a rootless [0] option, but that does require some sysadmin setup on the host to make it possible. That's a Linux kernel limitation on all container tooling, not a limitation of Docker.
> and that it keeps images between users in a database.
Not a traditional database, but content addressable filesystem layers, commonly mounted as an overlay filesystem. Each of those layers are read-only and reusable between multiple images, allowing faster updates (when only a few layers change), and conserving disk space (when multiple images share a common base image).
> I want to move things around using mv and cp, and not have another management layer that I need to be aware of and that can end up in an inconsistent state.
You can mount volumes from the host into a container, though this is often an anti-pattern. What you don't want to do is modify the image layers directly, since they are shared between images. That introduces a lot of security issues.
> Docker is still maintaining the runtime on which orbstack, podman etc are all using, and all the cloud providers are using
I need to fact check that one. runc was donated by Docker to OCI a while back. And containerd was created under the CNCF from a lot of Docker code and ideas. podman is sitting on the RedHat containers stack, which has their own code base. Docker itself uses runc and containerd, and so do most Kubernetes deployments. Many of these tools go to containerd directly without deploying the Docker engine.
DevOps isn't a tool, but there are lots of tools that make it easier to implement.
DevOps isn't how management can eliminate half the org and have one person do two roles, specialization is still valuable.
DevOps isn't an organization structure, though the wrong org structure can make it fail.
DevOps is collaboration. It's getting two distinct roles to better interoperate. The dev team that wants to push features fast. And the ops team that wants stability and uptime.
From the management side, if you aren't focused on building teams that work well together, eliminating conflicts, rewarding the team collectively for features and uptime, and giving them the resources to deliver, that's not a DevOps failure, that's a management failure.
For example, you may isolate a specific customer to bare metal so an escape doesn't compromise other customers. But within that bare metal, you may run containers because they make it easier to work with a read only root filesystem that's also trivial to upgrade. You can also add on user namespaces and seccomp in the container to minimize the risk of a container escape. And then the application may have its own sandbox that limits individual capabilities and which API calls it can run.
Every use case is different, and some layers may not be available depending on that use case. But rather than picking one point on the spectrum, one should pick a list of technologies that best solve each use case.
The other thing I've noticed lately is a strong push to get non-programming questions off StackOverflow, and on to other sites like SuperUser, ServerFault, DevOps, etc.
Unfortunately, what's left is so small I don't think there's enough to sustain a community. Without questions to answer, contributors providing the answers disappear, leaving the few questions there often unanswered.
And my favorite project for inspecting layers is Jon's https://oci.dag.dev/.
2. Your software needs to be distributed with a license that is compatible with your dependencies. You can't add restrictions if your dependencies forbid that.
3. No one will use your project if it doesn't have an OSI license. It's not worth the time and effort to read every license and get it approved for use by the legal team. If you're doing anything useful, someone will make an alternative with an OSI license and the community will ignore your project.
1. Work two jobs until you burn out.
2. Quit your paying job and hope you have you don't go broke.
3. Scale back or quit maintaining OSS projects.
I think companies, governments, and societies could do a better job funding this work. But since this is a "tragedy of the commons" problem, I'm not holding my breath that this will happen before the public experiences a lot more pain from failures.
In this case, the words from the LLM have no genuine appreciation, it's mocking or impersonating that appreciation. Do the people that created the prompt have some genuine appreciation for Rob Pike's work? Not directly, if they did they would have written it themselves.
It's not unlike when the CEO of a multi-national thanks all the employees for their hard work at boosting the company's profits, with a letter you know was sent by secretaries that have no idea who you really are, while the news has stories of your CEO partying on his yacht from a massive bonus, and a number of your coworkers just got laid off.
I'd also avoid loading the result back into the docker daemon unless you really need it there. Buildkit can output directly to a registry, or an OCI Layout, each of which will maintain the image digest and support multi-platform images (admittedly, those problems go away with the containerd storage changes happening, but it's still an additional export/import that can be skipped).
All that said, I think caching is often the wrong goal. Personally, I want reproducible builds, and those should bypass any cache to verify each step always has the same output. Also, when saving the cache, every build caches every step, even if they aren't used in future builds. As a result, for my own projects, the net result of adding a cache could be slower builds.
Instead of catching the image build steps, I think where we should be spending a lot more effort is in creating local proxies of upstream dependencies, removing the network overhead of pulling dependencies on every build. Compute intensive build steps would still be slow, but a significant number of image builds could be sped up with a proxy at the CI server level without tuning builds individually.
I'm the author of one of those off the shelf tools, and the rules_oci decision here always struck me as a bit unusual. OCI is a relatively easy spec with a number of libraries that implement it. Instead of creating a custom build command that leveraged those libraries to be an efficient build tool, they found commands that could be leveraged even if image building wasn't their design.
It looks like rules_img is taking that other path with their own build command based on the go-containerregistry library. I wish them all the best with their effort.
That said, if all you need to do is add a layer to an existing base, there are tools like crane [0] and regctl [1] that do that today.
The reason other build tools typically pull the base image first is to support "RUN" build steps that execute a command inside of a container and store the filesystem changes in a new layer. If that functionality is ever added to rules_img, I expect it to have the same performance as other build tools.
[0]: https://github.com/google/go-containerregistry/blob/main/cmd...
[0]: https://www.datacenters.com/providers/amazon-aws/data-center...