Hacker News Reader

Comment by amelius | Hacker News Reader

> if there's multiple programs interested in the topic, how do you make sure they all get all of the messages, if they all listen to the socket, only one gets the message.

Well, you could listen on a socket for subscribers ...

(Basically a decentralized approach vs a centralized approach)

> OTOH, dbus doesn't transit over the network, so eog on a remote host can't communicate with your desktop environment, so that's kind of a pain.

Yes, and what I find mysterious is that dbus was conceived by the freedesktop.org project. How could they have missed this important requirement?

View on HN

> Well, you could listen on a socket for subscribers ... > (Basically a decentralized approach vs a centralized approach)

But who is going to listen on that socket for subscribers? When there's no owner of the resource, just some programs may want to send to it, and some may want to listen to it, you need someone to own all of that; and that's dbus, as I understand it.

> Yes, and what I find mysterious is that dbus was conceived by the freedesktop.org project. How could they have missed this important requirement?

My personal impression is that the freedesktop people have effectively abandoned network computing as X11 used to provide. Clearly, I haven't stepped up to do the work either, but both the single host / multiple user and single user / multiple host use cases are at best neglected, if not abandoned. It used to be feasible and useful to run a host with multiple X servers for multiple simultaneous users, and those sessions may have had a mix of local and remote X clients; but for various reasons, that doesn't work so well anymore.

Some of the lack of support is because addressing the full scope of everything is actually pretty hard. If you run networkmanager over remote X, which network is it expected to manage? If you have multiple users on a single machine with bluetooth and someone sends a file to the host, which user gets a notification?

That said, I think for many things where dbus fits, there's a connection to a graphical user session, and so IMHO, it would fit to coordinate in a way that all participants in the session could use, which for me would be through the X server (which already allows for communication between clients). And for things where it's really a connection to the host hardware, that should probably be through the host filesystem. And so some applications might need to connect to a message bus in both ways, and maybe sometimes tunneling --- an application may legitimately need to contact a system daemon on the host that controls the user's keyboard, but that might not be possible through X, or the host filesystem where the application is running. But that's super complex and weird, so it goes into the later pile, never to be solved.

Distributed RPC, i.e. dbus over the network, is an incredibly complex and still basically unsolved problem. To really do it right and guarantee correctness across systems (which may be in different states of working or failed!) you need a perfect message queue with exactly once delivery... but in practice implementing that is incredibly storage intensive (you'd have to store every message forever). You're basically asking for something as complex and difficult to run as kafka--it's not as simple as just send dbus messages over a network socket.

> Well, you could listen on a socket for subscribers ...

"I'm afraid to inform you that you have built a message bus with RPC features.

You had a program that needed to broadcast some information. You have built some simple mechanism using a UNIX socket. You send everything anytime someone connects to your socket and close the connection.

Then, your software grew and could provide various other things. It became impractical to send everything everytime so you needed some way to advertise what information your software provides, and receivers would send the list of what they need when they connect.

Your program grew again, and you needed people to be able to call your code to do some actions, so you added a way to call methods that can take parameters and to advertise them.

With the mess you ended up with, you felt the need to standardize how information and methods are represented and used. You designed a DSL to write schemas and some kind of code generator using this schema to automatically build interfaces for these methods and this exposed data.

You figured calling methods and requesting information through a UNIX socket directly is unwieldy, so you wrote a small library hiding this complexity to the caller and making everything more convenient to use.

Your program evolved again and you needed the callers to be able to receive events without having to poll. Your library now contains an event loop that the callers need to handle.

Your program kept evolving. Before the number of generated events and the performance issues it caused, you felt the need to add a mechanism to filter / subscribe to events to avoid flooding the callers.

Time passes, and your to had to write a new, unrelated program that also needed to communicate. You wanted to reuse the same mechanism. Callers needed to listen to multiple programs. Those multiple service providers could come and go and you noticed requiring callers to poll multiple UNIX paths to find out if there is a socket there was not practical nor efficient because of the syscalls and that it's better if they could tell some kind of daemon they are interested in a service and get informed when it becomes available. Therefore, you took the whole mechanism out of the first program and made it a daemon so callers had a central place to discover what is here, and providers to advertise themselves. This daemon of course needed to be run at system start, or at least in a lazy fashion, or the different actors would, again, need to do the UNIX socket path dance, which is not quite convenient after all.

Time passes again. In the bag of services you wrote, some are related to the system, and some are only relevant to a logged user. You wrote some kind of integration with your init system and your session manager.

I'm afraid you have built D-Bus."

In reference to the recent HN repost [1] of [2]

(edit: damned, I spent an hour writing this shit)

[1] https://news.ycombinator.com/item?id=37162898

[2] https://rachit.pl/post/you-have-built-a-compiler/

> "I'm afraid to inform you that you have built a message bus with RPC features.

No of course, not, I would use a shared object file to do it for me. I.e. all functionality inside a library instead of a centralized process that can crash or hang.

We were talking about bare UNIX sockets and their potential drawbacks. I think they don't really scale / answer to all the needs D-Bus fulfills.

Could you expand on your shared file idea? I'm interested. How do you implement this library without a central service? Note that in my humorous comment I still expose why I think a central program is necessary, now you can disagree with the use case of course.

It's quite simple, actually. There is no need for a central entity, just like computers on the internet do not need a central server to communicate. You might say that they need a DNS server, but the equivalent of that on a UNIX system would be the filesystem, where named sockets can live (or you could use X11 properties to communicate between desktop apps).

Anyway, I feel that all the microservices are making me lose control over my system (and systemd is quite literally the mother of all these microservices, hence why I bring it up here).

Reply on news.ycombinator.com