Hacker News Reader

Comment by Arainach | Hacker News Reader

Excellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enlightening to debug various apps and frameworks and discover just how long repainting can take and how many edge cases there are around slow apps, hung apps, and so on. Who knows, maybe someone there still has some of the malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.).

It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.

View on HN

>> malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.)

This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing.

Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).

> Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m...

This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently.

That said, recent discussion on that latest one does look promising so maybe it'll finally happen.

That isn't really what I was hoping for. IMHO it is wasteful for every application to remember its window placement and restore it. Not only does that require additional code in every app, but if there is some kind of policy change there is no way to make it consistent. In other words, at the compositor level one could use the previous position as a default and have an option to do something different if another window is already there. It's simply not the applications job to place itself on the desktop. There are probably use case for such self positioning functionality, but restoring to the last position is not one of them.

The application side of it is more for things like the multi-window setup of things like GIMP so that windows that are "docked" next to each other will stay that way past restarts. That's one of the reasons that the newer proposals are doing things with relative positioning between a zone or main window rather than allowing applications to place themselves randomly on whatever monitor or space that they want, interrupting whatever workflow is going on (which actually allows for security issues, i.e. a window pretending to be a password prompt putting itself on top of a browser or something to confuse the user). This also allows for new windows from the application to request that they're positioned next to any others so that related things stay together. This also apparently helps in a few cases where a single "application" to the user is actually multiple separate programs that get run by a main interface. Not as common in new software today but it used to be one of the ways that a lot of older software worked and there's still a decent amount out there apparently that are maintained that way.

Keep in mind that this is also a request by the application, not a requirement of the compositor to obey it. If there's not sufficient space where the application requests things then the compositor can just ignore it and do what it believes makes sense.

>The compositor has a frame to paint and it can't have to wait on applications.

This is the source of the visual artifacts this article is trying to prevent, however. Sure, you probably don't want to block resize for multiple seconds, but in general the compositor is very responsive. The app may not be. If you just let the window chrome resize as fast as you get, what do you do with the rest of the window? Leave it transparent? Draw a default black or white area? Both are very ugly and very noticeable in practice.

This is the part of the problem space I know the most about - for a number of years I owned window chrome on Windows (don't blame me for the 1px border and way-too-subtle shadows, but after being overruled by PM/design, yes, I was responsible for implementing them).

As far as custom decorations - that is a lost battle. Companies and apps want their special design language and will simply not build for your product/operating system/compositor/whatever if you don't give them that kind of support. Twitter and Facebook both want their specific shade of blue and their specific font. Adobe is....well, let's not talk about Adobe. Browser tabs in the title bar are a P0 requirement these days and anyone who doesn't support that will be laughed out of the room. Etc., etc.

IMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window.

Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.

That trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!

I think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.

I think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)

It's kinda interesting that macOS doesn't even care about this problem anymore. When moving or resizing windows on macOS the hardware mouse cursor slightly trails ahead of the window, it's not very noticeable because of the 120Hz refresh rate (meaning the framebuffer surface spends less time waiting for presentation, thus reducing rendering lag) - OTH the lag is still visible, and this sort of minor visual imperfection is quite un-Apple-like (at least for 'old Apple').

It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.

> It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete

Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.

But if we continued using subpixel rendering then 1080p and 1440p displays wouldn't look as terrible, and we can't have that /s

Unfortunately a low of newer displays (especially OLED ones) don't have regular horizontal RGB/BGR layouts anymore. I have seen both vertical subpixel layouts as well as triangle layouts or other funky subpixel arrangements. Even if you configure/update your software to those different subpixel layouts the result is not going to be as good.

It's pretty noticeable on older apple hardware running newer macOS. Definitely a lot of compositing performance regression in Sequoia specifically.

Reply on news.ycombinator.com