Docker was basically a commercialisation of lxc and now we got nspawn going full circle.
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
Developers are a tough market to please, especially when your business revolves around open source software.
Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.
In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
I always assumed it was the same internal tech, now productized. Then of course, it became THE tech, overtaking vagrant and anything else for local development and deployment. The latter was the true money shot. Server runs exactly what you test with locally.
That's because there was never a proper sysadmin on your team to slap some sense in your decisions.
You've got too opinionated[0], too tied on yourself[1] monster which required too much resources what you can handle alone.
And your market is not the developers but the companies where whose developers have a daytime jobs: https://news.ycombinator.com/item?id=34377674
BTW just for giggles opened hub.docker.com with Network Tools opened:
368 requests, 9.62MB, Finish: 4.50s.
For a thing which should be a static cached copy with everything bundled. Sure, it's a progress from the previous iterations when it could hang for 30 seconds before actually loading... but this is still ridiculous.
[0] https://news.ycombinator.com/item?id=46739153
[1] Eg: the syntax to point the image to a specific registry is something... bad. So bad I just abandoned running my own registry, it wasn't worth it.
PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
TRANSACTION START
RUNBATCH set -eux
# TODO find a better userspace architecture detection method than querying the kernel
arch="$(uname -m)"
case "$arch" in \
aarch64) gosuArch='arm64' ;
x86_64) gosuArch='amd64' ;
*) echo >&2 "error: unsupported architecture: '$arch'"; exit 1 ;
esac
curl -fL -o /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch.asc"
curl -fL -o /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch"
export GNUPGHOME="$(mktemp -d)"
gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc
chmod +x /usr/local/bin/gosu
gosu --version
gosu nobody true
TRANSACTION END
RUN set -eux; \
# TODO find a better userspace architecture detection method than querying the kernel
arch="$(uname -m)"; \
case "$arch" in \
aarch64) gosuArch='arm64' ;; \
x86_64) gosuArch='amd64' ;; \
*) echo >&2 "error: unsupported architecture: '$arch'"; exit 1 ;; \
esac; \
curl -fL -o /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch.asc"; \
curl -fL -o /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch"; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
chmod +x /usr/local/bin/gosu; \
gosu --version; \
gosu nobody true
>>[0] 'Bridge' was and still is an established network term for joining two broadcast domains into one. Why the hell you decided to name your NAT'ed network layer a 'bridge'?
Like they told you 9 months ago, the terminology is exactly right. They're called "bridge" networks because they use Linux `bridge` devices to bridge `tap` adapters of all containers attached to the same Docker network.
There's nothing exotic about this terminology at all and complaining about it betrays your own lack of knowledge.
> PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
Do you mean Dockerfiles? RUN command has had heredoc support for 5 years now.
Oh, well. I guess you ride to your work on wheels, right? Because while you sit in the car/bus/train it's the wheels what actually rides on the road/track, right?
And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
You see, if they just called it as it should, ig:
bridge => hostnatnetwork // or internalbridgenatnetwork or whatever else
macvlan => bridge
But the last 1% is the people who actually makes the workspaces for these developers where they actually do run their fancy-shmancy containers. And for that 1% it surely pays naming this as they are.
And there is also that dumb decision to silently rewrite 127.0.0.1 to Google DNS without a question. Especially when 98% of Docker users are incapable to diagnose it by themselves.
So it's not only the naming of a network functions.
But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
> RUN command has had heredoc support for 5 years now
So ten years late? Adoption is off the limits, as you can see - that snippet is from the official Docker image.
And if I need to run multiple commands and not just one RUN statement? Like the people are asking for years? And you so knowledgeable person of course do know the reason for that, right?
Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes learning and communication harder by introducing ambiguity and indirection. "bridge" now means multiple things and not only is it needlessly obfuscated, it's wrong (explained later). I _hate_ projects that do this.
> Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
Rhetorical question, maybe it'll help you find a source of misunderstanding: Why do you expect it to join the broadcast domains of your containers and your LAN? And which LAN, exactly?
> No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN. Therefore you incorrectly expect and demand that term "bridge" accurately describe the logical connection between your container and your LAN.
Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
A Docker 'bridge' network is not necessarily routed to LAN either so your suggestion of calling them "hostnatnetwork" is wrong. Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
Disabling NAT for a specific brige network (com.docker.network.bridge.enable_ip_masquerade): https://docs.docker.com/engine/network/drivers/bridge/
Disabling auto-configuration of all firewall rules: https://docs.docker.com/engine/network/packet-filtering-fire...
> But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
All of this networking terminology is consistent and applied correctly but for some reason it's not clicking with your previous knowledge and you're trying to fit a square peg through a round hole. Take a step back, investigate.
> So ten years late?
Don't shift the goalposts. You claimed it still doesn't exist.
The docker mental model isn’t crazy different. Design choices made it unnecessarily less compatible with existing mental models. This probably contributed to the feeling of it being revolutionary, but impeded its adoption with paying customers.
In my experience adoption was primarily driven by developers and sacrificing clarity would hurt adoption. This perfect familiarity was never achievable anyhow. Docker was a completely new paradigm in software development, similar to how modern cars aren't just horse carriages without the horse. It transformed everything from development, testing, deployment, updates.
Experienced sysadmins often tried to approach it with "VMs but with less overhead" mentality, which couldn't possibly work. Coworkers would ask me how they can install SSH daemons in containers to update packages and view logs, or they would refuse to adopt Dockerfiles, sometimes setting up the container manually and exporting the image. They would wonder how to install many different services in 1 container and handle crashes, etc. It was a huge shift.
> Design choices made it unnecessarily less compatible with existing mental models.
Again, I disagree. Whose mental model do you prioritize? How invested are you in making Docker work just like whatever elaborate system those enterprise admins used before?
Let's not forget that by creating new abstractions, you're penalizing users who already possess foundational knowledge of the primitives. If you do this then I'm probably going to get a whiff that you care more about enterprise sales than developer experience (duh) which is a huge red flag for me.
Sorry, but you do insist on naming 'docker' bridge a bridge because it's underlying tech is a bridge. Make up your mind.
> A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
And noone absolutely ever cared about what it's a bridge down there. It could be using gronklew and maybe 20 people on the planet would care to learn what it is.
You are quite bent on it, but nobody cares how the containers talk between each other.
> Why do you expect it to join the broadcast domains of your containers and your LAN?
I expect a setting named 'bridge' in some $thing to actually implement a bridge.
Just like you pressing an AC button in your car expect a very specific thing to turn on/off and not 'Artifically flavoured windshield wipers served in a Container'.
Why do I want it is on me. Maybe I need it. Maybe I control my network and freely give out addresses to anything. Maybe I don't run on some cheap VPS somewere with only 1 IP I can't choose nor change. Bazillion of options. None matters because this is not about 'why do you want it'.
> And which LAN, exactly?
The path of a least automagik.
Only one ethernet interface? Then connect to that ethernet interface. Bitch in the logs what automagik was used and this can lead to the problems.
Many interfaces? Just shutdown (least pain for newbies) or start without attaching (harder to troubleshoot for newbies). Bitch in the logs in either case.
> Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN
'Your misunderstanding seems to stem what there are some people assume what they are driving in like c-a-a-r. When even 5y.o. knows it's the wheels what drives you around.'
> Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
Yes, exactly. Don't call it 'bridge' ~20 years ago and we don't have this talk here.
> Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
It's not even a property of a bridge network. For someone claming knowledge superiority this is a quite lame statement.
> links
Really.
> previous experience in an adjacent area where
Ah, yes, irrelevant 'adjacent area' of computer networks. I'm out of words.[0]
> All of this networking terminology is consistent and applied correctly
No. Go drive your wheels home.
> Don't shift the goalposts. You claimed it still doesn't exist.
I'm just mentioned RUN as an example. You decided what multiline is the goalpost.
And yes, if even the official Docker images doesn't use RUN heredocs then... there is something wrong with them. Like being implemented too late.
[0] but perfectly explains your previous 'flawed assumption'
Just like nobody ever cared that the container isn't bridged to their LAN in the first place - in fact the isolation is a feature.
But you should care, it's only the most common building block of network isolation for containers/VMs on Linux.
---
I'm not going to get into it further. I tried to explain nicely but you seem hell bent on complaining and not learning anything new. At this point the appropriate response is: RTFM, it's all in there, their docs are great.
If you don't have a clue about how things work under the hood and if you have no interest in learning anything, then stop complaining about imaginary technical inaccuracies.
If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.
I think you are the one who built an alternate layering system? On mobile, don't like to search rn.
If yes the basics are in the first comment- just not limited to the one RUN statement ofc. And if I do remember how the layers are made then making it transaction-like would be a great dedup saver eg for adding a static rarely changed artifacts - there could be essentially a ~one layer for how many you want images.
Implementation wouldn't be so difficult - you just stop committing until TRAN END; but the main is too provide a mental model for a not very experiencied dev how this supports his work and makes it more efficient on the storage and execution.
bullshit, you've been cunty right out of the gate since your first reply. I'm just surprised anyone else in the thread had as much patience as they've shown here.
you seem exactly like the type of person who is incapable of realizing they're a complete asshole and wonders why people choose to avoid them.
Now, considering you don't even know me: go in the bathroom, look atvthe mirror and repeat your comment.
Come back when the enlightment hit you.
lol What on god's green earth does this have to do with commercializing Docker? Do you really think better lighthouse scores on hub.docker.com was what was holding them back?
> what was holding them back?
Did you miss 'up to 30 secs' in the previous comment? It's not about lighthouse scores. It's the whole feeling of a hodge-podged project which is run by people who are - maybe - passionate but ultimately lacking in the basics.
The ultimate buyers of their product was the system operators and the system owners - not the developers per se. They failed hard everything that space. Having a dog slow the main content site didn't added anything positive.
Right, which is why bringing upva meaningless stat like how many assets their web site downloads feels like satire.
yes, and no. Probably both. Docker would not become such a default standard if it wasnt for all the open sourcing.
I went down the rabbit hole of Incus and once I realized it requires the full network aspects to be configured, then I figured using KVM, QEMU, Libvirt was more practical. Then stumbled into nspawn and it was basically ready to go.
Although, I haven’t done enough research to know if Incus provides certain useful capabilities.
It includes project-level resource separation, advanced networking, clustering, snapshots, and more. Plus, Stéphane Graber is super responsive on the forums and issue tracker. I bounced off of Proxmox, and can't recommend Incus more for hypervisor-type scenarios.