eliribble

Weird editorialization. I included a TL;DR at the top, you could have just copy-pasted it.

"Docker-compose is a tool for working with Docker containers. It solves very real problems with deploying complex applications. By itself it is not enough to make self-hosting applications simple enough for the mass-market."

Yeah, I agree, we're going to need a really compelling use-case not just for end users that run the application, but for the application developers as well. Nobody wants to maintain 3+ extra deployment files for the various also-rans competing with docker-compose.

What do you use to manage all those compose files? Do you have off-site backups? I'm constantly reading and re-writing docker-compose and bash scripting everything to fit in with the rest of my infrastructure it'd be good to hear about someone with a better way.

Oh, hey, holy shit! You're one of my heroes. I've read through your internal discussions on protobuf within Google, you did amazing work there, and held your own against a difficult political environment.

It sounds like you have no criticism of the technical approach, then, but rather just the business mechanics? That's eye-opening, given how much has changed in self-hosted deployment since Sandstorm started. If you started something similar today, ignoring business needs, would you build something technically similar?

Looks like https://dockerswarm.rocks says that the site is deprecated.

https://dockerswarm.rocks/swarm-or-kubernetes/ says "it's not sensible to build a new product using Docker Swarm Mode"

Interesting, I wasn't aware that Traefik could do this without significant modification to the docker-compose configuration provided by the application developer. I also thought that Traefik required some sort of higher-level container orchestration like Docker Swarm or Kubernetes.

I'll have to look in to that.

These are great points, and probably worth their own blog post to answer.

> First of all, why is this a concern? Idle threads have basically no impact on a system, so this generally isn't a concern

Idle threads have very low impact on CPU utilization, probably, if the application is well-behaved (and I expect most databases and caching layers to be well-behaved in this way). The application itself, however, will need memory and the way containers are built prevents the usual de-deplication of system libraries.

> but generally it won't even make the list of top-100 performance bottlenecks for most applications

True, but it makes the short list of "how much stuff can I run on a $100 computer", and it's one of the relatively few concerns an application operator has when they are not the application developer.

> Even if you wanted to enforce your applications sharing a postgres instance under the hood, why would you want that to be black-magic performed by the container orchestrator?

To make self-hosting much simpler. If the container orchestrator doesn't do it, what do you think should do it?

> Other stuff like DB backups just don't seem like issues docker compose users have. If you need to orchestrate across multiple nodes in order to meet your SLOs, then don't use docker compose.

The DB backups are meant for disaster recovery rather than supporting multiple nodes. I guess that's multiple nodes through time... But, yeah, I agree, docker-compose is not a good fit.

> Finally, it seems like the actual solution is significantly under-discussed. I both have tons of questions about how it's supposed to work, and I see lots of shortcomings with the parts that I do understand.

Yeah, agreed, I'll be writing other things to discuss what I think the correct solution should be. I'm curious to find out if other people have existing solutions to the problems I outlined. If it's a solved problem and I just don't know about it, that'd be better.

> I'd be interested to read an article which tried to articulate why such an opinionated API would improve SDLC-considerations over docker-compose, but I don't think that's the article I just read.

It is not, and you're right, it needs discussion.

While a bit of a hot take, you're not wrong. We need something that's less scalability focused than Kubernetes/Mesos/Docker Swarm but that doesn't put too much burden on application developers. Something that focuses on being secure, reliable, and understandable, in that order. I'm not aware of anything going for that niche. That means a new tool is in order.

> The way they frame that pihole example ("Whew! That’s a lot of stuff!") is just silly.

Yeah, you're probably right. Originally that line was in there when I had a breakdown of what each line in the docker-compose was doing. My editor thought that was unnecessary - it's unlikely people reading the post would need that kind of breakdown. So I rewrote parts to assume more baseline knowledge. I should have noticed that line and taken it out.

You're right about what we're trying to do, and I agree that the post doesn't really help someone be successful today deploying things. The post is more meant to gauge whether or not I'm alone in having pain deploying a couple dozen services with docker compose on a single box.

I want more people to have the power to host their own services. I think we can do that, but we have to figure out the right thing to build to do it.

Thanks, interesting.

Sounds like your container has some kind of side-car that makes it directly addressable over Wireguard without needing to address the host IP. Does that mean you'd need to modify the docker-compose in some way before `docker-compose up`?

How do you know which port Paperless is using for HTTP? When you want to load up Paperless in a web browser, are you typing in a service name, or the container IP address? If it's a service name, how are you doing DNS? Do you have TLS?

> I would say if the author was actually interested in solving this problem in a productive way they should first try to see if docker itself is amenable to altering their constructs to provide optional higher abstractions over common concepts via the compose interface natively.

docker-compose is a lot of things to a lot of people. When it was created I doubt anyone realized it would eventually be the de facto standard for deploying to homelabs. It's an amazing tool, but it could be better for that specific use. I don't think that segment is important enough to the team that maintains it to warrant the change you're suggesting.

Yeah, we're very early building this, the blog post is just a way for me to organize my thoughts and start fights online. It's, uh, embarrassingly useful to yell semi-coherent thoughts into the void and have experts yell back with a decade or more of experience and information about tools I haven't heard of.

> I'm quite skeptical that adding a layer of abstraction and switching to TOML instead of YAML will suddenly enable those scared away by compose to start self-hosting, but kubernetes and docker swarm were never in the cards.

Yes, this is an excellent point. I did not articulate it well anywhere, but the goal is for users to have something more like Sandstorm, with a UI to install things. The TOML is for application developers, not end users. It'll either go in a separate database or, ideally, in the source code of the applications to be installed similar to a Dockerfile. I haven't started yet, but eventually we need to work with application developers to support things they want and to make it easier to treat Tealok as the "easy option" rather than docker compose.

I'm actually a fan of Sandstorm, and think it got a lot of things right. I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Personally I think that it put a bit too much burden on application developers since it required them to develop applications specifically for sandstorm.

> I'm skeptical that what the non-technical self-hoster needs is a TOML DSL that abstracts away ports

I fully agree, the end user would not be writing TOML DSL files. The end user would get something much closer to an app store, or what Sandstorm did, with one (or a few) click installs. The TOML DSL would be written by developers familiar with the application and stored either in a separate database, or ideally in the applications source control like a Dockerfile.

Hey, blog post author here.

I'm curious how that last sentence was going to end.

Let's say I agree with you and that TLS termination is not a container orchestration responsibility. Where does the responsibility of container orchestration start and TLS termination end? Many applications need to create URLs that point to themselves so they have to have a notion of the domain they are being served under. There has to be a mapping between whatever load-balancer or reverse proxy you're using and the internal address of the application container. You'll likely need service discovery inside the orchestration system, so you could put TLS termination inside it as well and leverage the same mechanisms for routing traffic. It seems like any distinction you make is going to be arbitrary and basically boil-down to "no true container orchestration system should care about..."

In the end we all build systems to do things to make people's lives better. I happen to think that separating out backups and managing ports as an exercise for the deployment team raises the barrier to people that could be hosting their own services.

I could be totally wrong. This may be a terrible idea. But I think it'll be interesting to try.

> If he would have done so, in the very least he would have eventually stumbled upon Traefik which in Docker solves absolutely everything he's complaining about

I'm aware of Traefik, I ran it for a little while in a home lab Kubernetes cluster, and later on a stack of Odroids using k3s. This was years ago, so it may have changed a lot since then, but it seemed at the time that I needed an advanced degree in container orchestration studies to properly configure it. It felt like Kubernetes was designed to solve problems you only get above 100 nodes, then k3s tried to bang that into a shape small enough to fit in a home lab, but couldn't reduce the cognitive load on the operator because it was using the same conceptual primitives and APIs. Traefik, reasonably, can't hide that level of complexity, and so was extremely hard to configure.

I'm impressed at both what Kubernetes and k3s have done. I think no home lab should run it unless you have an express goal to learn how to run Kubernetes. If Traefik is as it was years ago, deeply tied to that level of complexity, then I think small deployments can do better. Maybe Caddy is a superior solution, but I haven't tried to deploy it myself.

This is a really good conceptual model, the tradeoff between flexibility and constrained declarative frameworks. The goals is to make self-hosting applications extremely easy and extremely reliable. With that as a goal, being highly constrained seems like the way to go.

I assume you're saying this as a container image author, not as someone who is deploying containers. It'd be great if every other container image author standardized on something like this. We just need someone to create the standard and some tools to make the standard a well-paved path.

How many steps is it for you to add a new service to your system? Say you wanted to try out Paperless NGX, what would you need to do?

Blog post author here - that's the right company name, but not the right company website. Our company doesn't have a website yet.

Looks like we may share a company name with some grifters. That's what we get for making our name an obscure mythological reference.

My blog post, I'm interested in whether or not others have had similar issues or have better solutions than I do.

Just because tax enable those things in the United States right now does not imply that is the only way they could be provided

I think this project is great, but I tend to agree with others - it's going to be somewhat doomed by second-system effects and the fact that coding is a social art form. You have to get other developers to buy in to what you're doing to be successful.

So what I wonder is this: is there a migration path instead? If the goal is to ultimately work on ASTs instead of treating software as a giant string, is there a way we can make incremental transformations to languages to move in that direction.

So far what I've considered is this:

1 - Automated formatting such as gofmt which eliminates any decisions around whitespace to culture people towards treating code as data rather than prose

2 - Language-aware editors that constrain user input to valid states based on 1

3 - Plugins for version control systems to optionally do diffs on ASTs rather than plain files

4 - Compilers that can operate on ASTs instead of strings

5 - Checking in ASTs rather than strings based on 3 and 4

6 - Updating editors 2 to operate on ASTs since 5 makes it reasonable to always work on ASTs rather than text

7 - Creating new styles of code editors that visualize and manipulate ASTs in ways that are independent of their text representations

There's probably steps I'm missing. I'm only just getting in to this space so I'm ignorant of most research that has been done in this area.

It can bring a cost savings, but the savings is very slight if you're comparing apples-to-apples. For example, if you're using RDS in a multi-AZ replication set up with periodic S3 backups you have to take into account running each of the replica instances and the S3 costs. What we found was that RDS was treated like a black box to our automation (which was a good thing for a long time) and so we were less attentive to being over-provisioned and actually testing failover situations than we should have been.

Yes, we did move to a regular EC2 instance for our cloud service. Where we install databases on premises for our customers it can be bare iron, but it can also be a similar virtualized setup.

Maybe Yani himself can chime in when it's daytime in his time zone, but if memory serves correctly he did look at WAL-E. It had some nice features in its favor. I'm not sure why it didn't make it into the blog post.

I'm the decision maker for the company in question - Authentise. It was a combination of cost and the fact that we need to support an in-datacenter install option for our customers. RDS is really great if you need all of the failover and backup options it provides and you don't have the manpower to invest in it. That's how we got our services started. But, as our expertise and automation grew it ended up being a modest cost savings to handle our own backups and redundancies. Primarily, though, we just can't install RDS in someone else's datacenter on premises.

Original author here. I wasn't paid for the article. I wrote it primarily because many people know Github and I've been quite happy with Bitbucket as a replacement. I had a good experience for our use case and felt like sharing that.

You raise an interesting point about justifying my choice of Jira. Since the topic was comparing SCMs justifying Jira seemed tangential. Perhaps I will explain in another post what our justifications are for using Jira.