Hacker News Reader

Comment by yamtaddle | Hacker News Reader

Redhat-steered projects act an awful lot like they're trying to make it really hard to compete with Redhat without picking the same projects they do—so, being at their mercy, unless you have a lot of resources to maintain forks against a maybe-hostile upstream. Been like that at least since the whole Systemd thing. Maybe that's not what they're doing, but it sure looks like it.

As a user I find this frustrating because I actually wouldn't mind some consolidation in Linux, especially in the GUI layers, but tend to hate RedHat's taste in these matters.

View on HN

It is essentially the same "embrace, extend, extinguish" policy that microsoft used to use. The fact that the underlying code is open source apparently doesn't matter too much. If someone else did come up with a compelling alternative init system just make systemd a soft dependency of your desktop environment, and make sure that patching that out requires a lot of active maintenance and ongoing work.

I never noticed they pulled the plug on systemd for the extinguish, when did they do that?

And how exactly was any of it an embrace? Redhat was a Linux distribution provider from the start, they didn't take something over in order to destroy it. They just created systemd and provided help to distributions if they were willing to use it.

The distributions were fully aware what they were doing when they abandoned Sysvinit, it wasn't a backhanded tactic by redhat.

A recent example of something close to EEE is vscode, which is now getting "refactored" to utilize non-open source plug-ins for everything, effectively extinguishing it from a free perspective.

That's not how EEE worked. From wikipedia

1. Embrace: Development of software substantially compatible with a competing product, or implementing a public standard.

2. Extend: Addition and promotion of features not supported by the competing product or part of the standard, creating interoperability problems for customers who try to use the "simple" standard.

3. Extinguish: When extensions become a de facto standard because of their dominant market share, they marginalize competitors that do not or cannot support the new extensions.

Systemd is part of "Extend", and what they're extending is the linux ecosystem. Sure, those extensions might make things better in the short term and for some uses cases, but they're now owned by IBM so good luck!

Did you even read what you've quoted there?

Redhat didn't embrace a competing product, redhat has been a Linux distribution from the start.

They extended it by adding systemd, yes. I said as much before too.

They didn't extinguish Linux as their extension continues to be open source and ready to fork.

If you're going to argue from the perspective of sysvinit instead then neither embrace nor extend happened. Whatever redhats goal were, it wasn't EEE

> Did you even read what you've quoted there?

Yes, and I think it's pretty obvious to anyone who is familiar with the history. EEE is about a set of anti-competitive practices, ones that are definitely and obviously being used here. Whether they do that with malice aforethought is unknown, and nothing there is illegal by itself, but it is a pretty clear thing they're doing.

Microsoft didn't embrace office document editing software, microsoft word was one of the first document editing suites. And yet that is one of the examples in the wikipedia article: https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

Did microsoft extinguish document editing software? No, they just used their market position to make sure that their implementation was the most dominant.

Another key example is "Breaking Java's portability". Do you want to talk about how the move to systemd has effected the various BSDs? Here's a talk about it: https://papers.freebsd.org/2018/bsdcan/rice-the_tragedy_of_s...

> extending those standards with proprietary capabilities, and then using those differences in order to strongly disadvantage its competitors.

But the one qualifying bit for the EEE strategy to work is to remove the open source option, replacing it with something that can only be provided by them. And systemd is open source as far as I know, So the extinguish phase isn't possible in that sense.

>But the one qualifying bit for the EEE strategy to work is to remove the open source option, replacing it with something that can only be provided by them

Well if that's the definition you're using, why are we even talking? 6 comments deep and you're only now bringing up that by the definition you're using only proprietary software can use EEE? That's a pretty big point, and probably should have been brought up in comment number 1.

Like what are you even talking about if you're just bringing that up now?

And I mean sure, define EEE in a way so that it can only apply to proprietary software. However in my first comment I said this: "The fact that the underlying code is open source apparently doesn't matter too much."

Personally, I think you can use the techniques of EEE to get control of an open source project, and whether your alternative is open source or not doesn't really matter since what it's actually about is control. Making sure the open source community treats your solution as the de-facto standard, making sure that your solution with all it's eccentric little commands is the one taught in schools, selling ad-on projects like log-collections daemons and making it harder for your competitors to do the same.

https://slatestarcodex.com/2014/11/21/the-categories-were-ma...

EEE is descriptive of what's happening, it's not prescriptive. If you'd like I can say that "redhat is doing stuff that looks a lot like EEE except for this list of caveats which make it slightly different from these other cases, but not different from these cases". But if we're done debating definitions we can talk about the actual behavior I find objectionable, but that's all pretty well documented in other places.

That definition was quoted from the first paragraph of your source, Wikipedia.

And no, it's not descriptive of what is happening. Systemd was disruptive, yes. But not all disruptive things are EEE.

>but not all disruptive things are EEE.

I'd agree with that, there are no rock solid definitions we're going to agree on. Still, there are a number of places where a small concession from Redhat would have made other people's lives a lot easier. See also, this very thread and support for server-side-decorators. A decision made by someone directly employed by Redhat.

Innovation without compromise that significantly impacts downstream software, and where patches aren't accepted. That looks a lot like using a market position to unduly influence other projects to me, and they'd get a lot more sympathy if they at least said "pull requests welcome" or if Redhat paid an engineer to make libdecor less awful. (Note that redhat is also paying developers for libdecor development work)

There are a lot more examples like that floating around.

> Still, there are a number of places where a small concession from Redhat would have made other people's lives a lot easier.

And I completely agree with that. I just took an issue with the EEE reference

What's Red Hat got to do with this? Sure, they're a big GNOME contributor, but so is Canonical or Purism or Endless or, until recently, System76.

And that's always the rub, isn't it? It's next to impossible to prove that most of the people in key decision making positions work for Redhat or are Redhat alumni.

Redhat is the main force that created systemd. And so their heavy-handedness into Wayland isn't surprising.

That's a complete non sequitur. They mishandled A so necessarily they must have mishandled B as well. What kinda argument is that?

And systemd is controversial in that a vocal minority says it was a disaster. I don't think it is, definitely not compared to Wayland.

Have you ever worked with RedHat? This is just how they are, across the board. (And maybe now we should be saying the way IBM is, who's history is even worse...)

> And systemd is controversial in that a vocal minority says it was a disaster.

Not a minority, and some of the loudest voices are the experts whos opinions matter most.

I am one such expert. Most technical people I speak with about systemd are largely ignorant as to the design considerations.

Yeah, no. Most actual experts very much prefer systemd due to it actually solving the problems inherent in booting up a system with all the services, and resolving the dependencies between them, instead of just being hacky bash scripts that only work on a single machine (if at all).

The only people who have problem with systemd is “old-timers” that just refuse to learn anything new, and just blindly believe that the old status quo was better.

That's the kind of attitude I often see from people who have never spent time debugging a dependency resolution issue in startup actions. People who do not have significant experience in running large scale, reliable systems. People with a desktop focus, who know little about systems internals.

Engineering reliable systems often means reducing complexity. A major unaddressed complaint regarding systemd is that there actually isn't a need for dependency resolution during startup, at all. Dependency resolution is very complex and introducing it makes testing changes far more difficult. It is not a desirable feature for most server applications where reducing complexity and increasing operational visibility take precedence.

> The only people who have problem with systemd is “old-timers”

You may be confusing experience with age. The two are related but they are not the same.

> that just refuse to learn anything new, and just blindly believe that the old status quo was better.

This is just silly. There are all sorts of wonderful features related to containerization, cgroups for job control, structured invocation for processes, etc.

The primary issue with systemd isn't that any of these features are bad. The issue is that they have been conflated in a mostly unspecified, monolithic system which discourages composition, inspection, and simplified operation.

It's a big ball of mud architecture. You can't use the service invocation components standalone. You can't use the RC components standalone. You can't use the init components standalone. All of these things should be possible, but aren't, and that is the core issue.

https://en.wikipedia.org/wiki/Big_ball_of_mud

Thanks for the level-headed answer, I’m sorry I lost my cool there, but all too often one can run into Wayland/systemd-haters who are like absolutely unreasonable.

What I compare systemd to is the previous iterations of linux init systems, and compared to them it is a huge positive change in my, and most of the linux world’s opinion (it was voted on multiple times in probably the most democratic process at debian, was decided independently at several other distro). It solves real problems (e.g. pre-disk mount logging) in a relatively modular way, and I think we should cut some slack here, because often times in complex problem domains a monolith is the correct choice. Multiple different tools will actually result in more complexity due to coordination, and I would go as far as that was part of the problem with previous init systems.

May I ask you what would be your preferred solution in place of dependency resolution at runtime then? I do agree that it is “one more moving part”, I just fail to see how else could it work.

I wouldn't do dependency resolution at runtime at all on servers. I would do it at install time and generate a static set of ordered steps. This has a few desirable effects:

* It ensures changes can be audited, observed and tracked (or caught and prohibited). Explicit change control points are key.

* It provides an interface and an option, divorcing the dependency tool from the system which invokes the ordering

* Ordered sequential steps takes longer, but adds determinism. It enormously simplifies the startup process, including auditing where things go wrong in logs after the fact. For servers, this is far more important than lower start times.

Most systems end up with a distro/systemd controlled base and a custom application on top. Often there's all kinds of special harness around the application, far beyond what can be fit into systemd. So there's always necessary duplication and struggles around interfacing different kinds of systems.

For custom applications, I tend to advise that the development team first come up with the success criteria for their app. What conditions would they page on, and what runtime dependencies does their app need? (Databases, up to date feeds, kill switches/feature flags, etc). I suggest that they re-use their monitoring code to also control orchestration of service startup. For example, not starting a front-end app unless a connection check to a backend SQL server passes. In these cases, the orchestration may be significantly more complex than what can be performed inside of systemd -- and this is part of why a clean and extensible interface is so important.

Feel free to bring a better solution than systemd.

Talk is easy, building something that works for the majority is hard. And one thing for sure is that not everyone will be pleased

OpenRC

Haven't seen any of my clients use it in production yet.

No one ever got fired for buying IBM. But also you probably have used an openrc based distro if you've used docker, as alpine linux is very common.

Note that Red Hat's business model is charging money to be Linux experts. It's against their interests for non-Red-Hat Linux to be viable, or for Red Hat to be comprehensible without their services.

I mean, as an engineer, I don't need to assume malicious intent (not saying you do) to understand how this happens. It's much easier to support a single code path rather than options for everything. It's the same reason I tend to shy away from premature abstractions: every option is a drag on future code built on top of it.

Reply on news.ycombinator.com