Hacker News Reader

Comment by aroman | Hacker News Reader

This is a great positive example of the (often negatively presented) double-edged sword that is extreme backwards compatibility.

X11 is an incredibly backwards compatible system, but this comes at the cost of positively absurd code complexity. I remember reading some release notes from a new version a few years ago that said 100,000+ lines of deprecated code were removed, and that was barely a drop in the bucket.

And, of course, this is why projects like Wayland and Mir (Canonical's new display server) have been born — it's simply too cumbersome (and in some cases, impossible) to innovate on a behemoth like X11.

But people forget why X11 has such a giant codebase. It's not just bloat — a lot of it exists for a very good reason, and this article is an excellent illustration of what that is.

View on HN

I will never understand why people insist on making new clients and servers that are backward-compatible with terribly old, deprecated protocols, instead of just preserving the old clients and servers that spoke those protocols natively within some sort of sandboxed emulation system.

Take Trident (IE's rendering engine), for example. Why can only one version of IE be installed at a time? Why does IE11 come with IE7/8/9/10 "compatibility modes" (which, much as they might claim, aren't faithful recreations of IE7/8/9/10 behavior)? Wouldn't it make more sense to just ship an IE "browser chrome" that could load up the actual rendering engine DLLs from IE6-through-11 (similar to how Google made IE connect to Chrome Frame), with increasing levels of virtualization and sandboxing as you go further back to cope with the difference in environments?

Apple and Microsoft have both got this right a few times: Apple had Rosetta for PowerPC emulation; Windows had WOW16 and WOW64. But besides these isolated instances, it really isn't a common idea.

EDIT: I didn't see your edit which gave Apple's Rosetta and X11 as examples. Those are good counter-examples to my point. But then again, those strategies were not done by X11 itself, rather by compatibility layers as you suggested. And in fact this is precisely what a lot of companies are doing, for example the XMir project (http://mjg59.dreamwidth.org/26254.html).

____

I agree that that's a great strategy in general, but I don't think it translates well to display servers.

You can't just "put it in a sandbox". You need driver support, and you'd need a sandboxed environment sophisticated enough to deal with all the idiosyncrasies of the X protocol.

Even putting running an old version of X11/UNIX in a virtual machine is of limited benefit, because X11's core functionality is often tightly coupled with drivers and other things which would need a lot of special wrappers and hacks to work properly.

I'm not saying it wouldn't work (and I am not an X11 developer), but I do think there are extremely compelling and legitimate reasons for people to not follow your suggestion (at least in this case).

Why not make a clean break? None of this sandbox stuff.

X11 2: Electric Bugaloo - The list of major graphical linux programs in active use is very finite and most of it doesn't talk to X directly but relies on stuff like Gnome or QT which are libraries that have frequent churn anyway. No?

I bet if you could get the Top 30 projects to target your clean shiny X replacement in one fell swoop you'll end up most of the way there in terms of adoption within a few years. Linux is command line centric and frequently updated after all with users who expect things to break anyway. "Old" X odds and ends that won't be updated can be thought of much like Java Applets are thought of now a days.

Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays. Nobody expects to run 5 year old apps on their smartphones, but desktop linux can't afford a clean break once every 2 decades?

You're looking for the Wayland project. Excellent implementation. They reuse lots of ancillary X11 code too, like the input code from X.org, which has the virtue of working with a billion weird language/input/hardware combinations.

You can run X applications on top of it, transparently. They perform just as well.

GTK+ and Qt have been ported to it, Gnome and KDE are working hard at getting full ports over.

Canonical, of Ubuntu fame, has recently started working on a competitor, following the same strategy, called Mir. They've ported Qt to it, but are generally lagging. Wayland has the advantage of being crewed almost entirely by old X.org hands, who know a lot about what they're doing.

> Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays.

That was exactly what I was suggesting with "sandbox stuff"--except that the OS can provide a stub library that recognizes what that "abandoned Chemistry application from 2006" is asking for, and spin up the VM itself, in the background, making it look as if it was just a compatibility layer. See, for example, Windows 7/8's "Windows XP Mode" -- or, as I mentioned before, Rosetta.

That's not as easy as it sounds. In a desktop GUI, customers expect seamless integration between such sandboxes. Making the clipboard and drag and drop work two-way between such VMs can be a lot of work. Even for simple text, that may involve encoding conversions. That, in turn, means that the host must be able to infer what encoding the guest expects. Styled text and graphics are way harder.

The VMs also must react to changes in the parent OS such as a change in the keyboard layout, two apps running in different VMs may both want to have some control over hardware, etc.

The approach works well in Windows because Microsoft spends lots of time on it and because the compatibility changes aren't that great.

Of course, it also works well in systems where there is no need for the VMs to interact (other than via the network). VM (http://en.wikipedia.org/wiki/VM_(operating_system)) is a nice example there.

Making the clipboard and drag-and-drop work between any two X11 programs running on the same server is a hell of a lot of work, and many times simply impossible. So why are your goals for sandboxing so impossibly high?

Because I want my system to work, unlike that example you give. And impossibly? It worked fine in Apple's 68k-PPC and PPC-x86 transitions, across the various shims that Microsoft has for all kinds of old software, and may even work fine across various VM hosts running on Mac OS X (disclaimer: I have little experience with those)

> Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM.

No, just load a compatibility library, possibly in the form of a wrapper program. If Wine can run Windows programs on Linux by translating Win32 to POSIX and X11, which it can to a very usable extent, something equivalent can translate X11 library calls to whatever the new library speaks, especially since it will be able to use whatever X11 code it needs.

And now, you're supporting the old system and the new system.

Maybe it's worth it. Maybe it's not. Just keep in mind that if you're keeping an emulated version alive, someone has to mind the emulator, and that's not free.

Sandboxing doesn't remove the need to provide ongoing security maintenance for a Web browser engine. For one, sandboxes can be attacked through whatever IPC mechanism you provide (e.g. Pwnium). But more importantly, the Web browser engine enforces security mechanisms that OS-level sandboxing does not address (e.g. the same-origin policy and history sniffing countermeasures).

People get upset when stuff don't work. If you want customers, you make stuff work. You don't break stuff, or change stuff so they have to call you and have you tell them how to make it work.

How do you deal with security updates? You'd have to apply them to differen code bases all the time. Pas bad as it sounds it's easier to work with one for base

Joel Spolsky, Things You Should Never Do, Part I

http://www.joelonsoftware.com/articles/fog0000000069.html

  Back to that two page function. Yes, I know, it's just a 
  simple function to display a window, but it has grown 
  little hairs and stuff on it and nobody knows why. Well, 
  I'll tell you why: those are bug fixes. One of them fixes 
  that bug that Nancy had when she tried to install the 
  thing on a computer that didn't have Internet Explorer. 
  Another one fixes that bug that occurs in low memory 
  conditions. Another one fixes that bug that occurred when 
  the file is on a floppy disk and the user yanks out the 
  disk in the middle. That LoadLibrary call is ugly but it 
  makes the code work on old versions of Windows 95.

The right way to resolve such bugs is to have abstraction layers that hide these awful hairiness.

If hair regarding low-memory conditions, Internet Explorer and other things all appears in the same function, some abstractions are definitely missing.

Also, if the reason for these things isn't clear, that's what comments are for.

Rewriting code, from my experience, has had only great results.

This is so absolutely true. What nobody recognizes is that most of these really fundamental systems we run on - unix system calls, our window managers, our shells, etc - were designed and implemented in an era where the supercomputers couldn't dent your Nexus 5 in terms of most computing performance metrics.

They didn't have the luxury of having an API that might be a few thousand KB big loaded into memory to act as a single redirection to your underlying implementation. They worked in the confines of bytes of memory, not gigabytes.

We can afford to be generic, to make extensible runtime programmable interfaces and runtime evaluation type dynamism, because we have the performance necessary. But our core APIs are still written like its 1980.

However, this line of reasoning also leads to slow web applications that make a high-powered workstation feel like a slow 386 from 15 years ago.

My only concern is that over the years I've seen everything go in cycles. So maybe 8mb is not a lot of RAM for a computer nowdays. A few years ago it was a lot for a router. I like my OpenWRT router that runs Linux. Before that the same could be said for mobile computers with GSM modules (cellphones).

I'd like to think that Linux will continue to run on machines with at most 4mb of RAM with not much processing capacity because if history keeps repeating itself we're going to keep on inventing new devices with those constraints.

But our core APIs are still written like its 1980.

Exactly what level of abstraction is good for the "core APIs"? Something actually has to send a series of bytes to be written to the disk, even if a bunch of serialization and encoding abstractions are written on top of that. If the latter is the "core", what's the less abstract stuff that inside that?

Rewriting can be great as long as you take all the things Joel notes into account and decide it is still the right decision for your project.

Then you end up with "abstraction layer" like autoconf, and the "cure" is MUCH worse than the disease.

autoconf is operating within difficult constraints (only /bin/sh and make are assumed to be installed), and uses shell scripts that write shell scripts that write shell scripts.

A) autoconf is not similar at all to an API layer hiding hairiness in its domain

B) autoconf becoming terrible does not mean that portability abstractions are necessarily terrible

Amusingly enough, Wayland is already developing backwards-compatibility cruft even though it's not shipping anywhere yet. The core Wayland support for window creation and management cannot support minimizing windows, so it's being essentially obsoleted by an extension called xdg_shell, but xdg_shell isn't mandatory so applications and Wayland compositors will wind up having to support both.

This is funny. You know that window minimization is not in the X11 spec either, right? Its something that X window managers handle internally (just like on Wayland where this is an internal compositor thing).

Of course, desktop apps wanted to interact with minimization (read current state, etc), so the WM authors joined forces to create a spec that allowed this. See how it specs minimization here: http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html#id...

This is part of the "wm spec", and the thing corresponding to it in Wayland is called "xdg_spec", Both are produced by "xdg" (X Desktop Group) and are optional things that are used by "traditional desktop environments".

It is a good thing that wayland does not enforce the existance of window minimization, because it is not always something that makes sense in all use cases of wayland. For instance in a phone like Jolla.

>The core Wayland support for window creation and management cannot support minimising windows

One of the problems with Big Rewrite projects is that, inevitably, early releases will never get feature-parity with the latest version of whatever it is that's being rebuilt. Managing expectations then becomes critical to avoid this sort of legacy-support nightmare.

I thought things like xdg_shell grow up as either extension or in Weston, and then it gets merged into the stable Wayland protocol after it is battle tested and ready. Is this not the case for xdg_shell?

This is probably a stupid question but why can't someone make an X11 version that speeds up and optimizes everything but breaks backwards compatibility?

That's basically Wayland. If we're lucky, the KMS+Wayland+XWayland stack will end up with a better architecture, backwards compatibility, and less code than the old XOrg.

(My hope is inspired by ZFS which, due to its "rampant layering violations", had more features and less code than the UFS+LVM stack it replaced.)

Because something you depend on every day needs one of those backward compatible features.

There's really no point to doing that, since X is a protocol, and a pretty flexible one at that. Your new shiny can have an X server running in it to act as a go-between and talk something more modern with the new stuff instead.

The OP would have been able to do the same stuff with Exceed on Windows or the X server that comes with OSX, after all. X11's survival on linux seems to be largely a matter of momentum and the fact that it's the least common denominator in a fragmented landscape.

But now Ubuntu has approached the level of ubiquitousness that's necessary to push for a real change (thus mir) and the rest of the linux world is rallying behind wayland.

What are you going to run on it? 99% of userland programs expect compatibility with some historical version of X11. Are you going to rewrite GNOME and KDE and everything else?

Many userland applications don't use X libraries directly, but use a toolkit like Qt and GTK. Both projects are working on Wayland support [1, 2], bypassing X completely.

1 - http://wayland.freedesktop.org/qt5.html

2 - http://wayland.freedesktop.org/gtk.html

It's worth noting that GTK+'s abstraction over X11 is fairly leaky in places- hence the always shaky state of its ports to Win & Mac.

Qt has less trouble in this regard.

Reply on news.ycombinator.com