audidude

> VTE based terminals, Kitty and WezTerm are some of the few terminals that accept C1 controls by default, within UTF-8 encoded data. My recommendation would be to not use these terminals with untrusted data.

It seems the author didn't look at the code because in VTE, bytes enter UTF-8 decoder first and therefore the parser is already dealing with 32-bit unichars.

Interesting. I don't work on VTE but just landed dozens of patches with no issue at all from pleasant maintainers.

Dunno how long it's been since you used it, but I believe keybindings for this are disabled by default now and you have to toggle them on in your profile preferences.

I really doesn't matter to be honest, because this was just to inform the theoretical upper bounds as part of my work to make VTE faster.

Glad this Termkit prototype work is right on track then. Generated a 655MB lorem text file and it came out at 0m8.186s.

I've heard various things about VTE over the years including:

1. They "like" xterm because i get value out of screen tearing and feeling like they can catch what is scrolling by (even though it's literally seconds slower) 2. They don't like the 40 fps cap it currently has because it predates reliable access to vertical sync information in applications. 3. They are using the GTK 4 port which draws with cairo and uploads surface data on every frame update.

The patches I put together for VTE this week (while having some down time with Covid) address a portion of that by making it render fonts/rectangles/emoji/etc on the GPU using GTK's OpenGL renderer (which I also wrote a large portion of).

It sounds like you're pretty biased, and that's fine. You do you.

But I'm pretty sure that code pre-dates reliable access to vertical sync from applications. Either via XProperty notifications from the compositor, XPRESENT, or actual glSwapBuffers() and/or equiavalents.

Author of the code in question here ...

Not sure how this got off terminals and on to compilers, but here we are.

Having written the profiler I used to optimize this (Sysprof), and a large portion of the OpenGL renderer that GTK uses, I know a thing or two about what code runs in the hot path from VTE. So I could target something that does enough of that to see what should be possible as a best case / upper bound.

That's why I'm fairly confident that adding the rest of the features that VTE has (which is one of the most comprehensive terminal emulator code bases out there) wouldn't affect this particular test case.

That said, I have no interest in doing that. I'd rather just go make VTE fast at drawing and call it a day. I have literally dozens of other projects in GNOME that require my time.

What I was trying to show, and also why I have no interest in creating something out of Termkit, is just that we have a lot of low hanging fruit that can still be fixed. I'd rather do that work as part of VTE so all the applications I use benefit (including GNOME Builder, which I wrote and maintain).

This was a fairly straightforward implementation of VT parsing and uses the retained render tree in GTK properly. There is no need to sacrifice CPU for GPU or vice-versa. You can literally do better in both directions while rendering at full frame rates.

First we make it fast. Then we remove the FPS restriction. I would be very surprised if I don't manage to nuke that as part of this work.

It's still GTK 3 using VTE and so doing the drawing with Cairo. It (VTE) has a lot of optimizations that made sense when we didn't have working GPU drivers.

But for GTK 4, we can do things much differently and faster.

I already maintain two dozen GNOME projects. I wrote a huge amount of GTK's OpenGL renderer, the profiler we use on GNOME, GNOME Builder, and a huge amount of other parts of GNOME. I've even started to fix the library that people will actually use (VTE) instead of fragmenting communities with some other project.

My commitment to "FOSS" is not up for debate.

I've written in multiple places what it takes to get stuff that fast. If that isn't enough for any of those projects, I'm happy to take money in exchange for my time.

The example you gave, Audacity, has a flatpak managed by one of the flathub admins, and most definitely does not include telemetry.

https://github.com/flathub/org.audacityteam.Audacity

> Much of that finesse has been lost in this new generation of GUIs built on 3D APIs where the entire window is thrown away and redrawn every frame.

GTK 4 uses eglSwapBuffersWithDamage() scissor clipped to the union of the damage rectangles. Operations falling outside that clip are culled.

Only the damage areas are composited, assuming a competent compositor.

GTK 4, which the author mentions in their post, uses a retained rendering model so that the UI can be rendered on GPU. Widgets snapshot render nodes which are immutable and can even be cached between frames. Those render nodes are diffed to calculate the damage region automatically.

That damage is then used to scissor clip in GL before the render tree is converted to shader commands. The original damage flows through to eglSwapBuffersWithDamage() to limit the screen damage to the diffed areas. Some work is needed to do tiled/threaded processing of the render tree to reduce pathological cases instead of large scissor clips. Given how much a DAW changes per-frame when animating, it's probably close to pathological anyway.

In git, there is support for something akin to swap-chains so that you can use DMA-BUF textures (or similar, such as IOSurface), and specify the damage between each buffer. This is useful for things like virtual machines or other external plumbing where you have a buffer+damage to integrate into GTK's composition pipeline and don't want full-frame renders to result due to gsk_render_node_diff() falling over.

Currently, they are rendered using Pango (thus Cairo) to memory and then uploaded into a texture atlas. This happens once per-glyph/font/size (plus or minus some variants for x/y alignment).

You then use the "color shader" to copy from the texture atlas into the destination FBO to both create the shape and color at the same time. Since you often do this for a whole text editor, you can copy all of the glyphs and color them in a single glDrawArrays()s.

When GLyphy lands, it's a bit different. You preprocess the bezier curves into a simplified format (which doesn't have overlaps) and upload those to a texture atlas (but for general data). Then use GLyphy's shader program to render them, similarly to how the color shader was used.

For more general paths, which is much slower but also still very neat, you would do something akin to Pathfinder.

> but isn’t certain things faster to render on the CPU still?

It really depends on the context.

If you look at it in a certain light, fonts are just "vector graphics" and there are ways to do that without using SDF (signed distance fields) and get really high quality precisely positioned fonts fully rendered on the GPU.

An example of that is the GLyphy project and we have a branch which can render fonts using that with GTK 4. It's a bit slower, but better quality and still easily able to handle 144hz full screen text scrolling. It requires some preprocessing before uploading glyph curves to a data atlas on the GPU.

For other shapes, there are options, but given GTK has so few upstream developers, they aren't quite ready for prime-time so they'll likely be part of GTK 5 in the form of "GskPath" if you feel like browsing the branches with that.

> a lot of functionality is moving into a gnome-specific library (libadwaita)

Sort of?

Mostly just design patterns that are explicit to GNOME. And that is primarily a response to people being (rightfully or not) unhappy that too much GNOME was ending up in GTK itself.

> And as VS Code by popping up in more and more sites,

On Linux, VS Code can't even resize without glitching and showing unrendered portions of the window like it's 1999. Nor can it kinetic scroll. Those are two expected highlights of quality GPU-backed software.

Hopefully running in an actual browser would make this better simply because you can choose a browser that isn't Electron.

How does that make it "not desktop"?

This is literally the dumbest argument ever. It's one extra line of code. If you have the GDK X11 backend, get the XID and call MoveWindow() yourself. If you have a macOS Window, call [NSWindow setFrame]. I don't see why you expect GTK to be your abstraction for insecure UI practices.

The compositor should be in control of this policy, not applications. Which is why that API isn't there any more.

> And the same situation on Linux/GTK, system uses Cairo - pure CPU rasterization library.

GTK 4 renders by default using your GPU. The only part that is CPU rasterized is if we don't have an accelerated path for it (rare) and glyphs are first rendered on CPU, uploaded to a texture atlas, and then GPU colored/blit from there on.

GTK 4 has built-in support for at-spi in it's native controls and provides GtkAccessible interfaces which you can use to make custom widgetry accessible.

> I really don't see what Gnome's doing to comply with this that everyone else isn't.

That's because Sun and GNOME created those layers two decades ago for Xorg.

I have no idea which project's you're alluding to, but I'm certain they weren't on GTK 4, which is what this thread is about.

GTK 4 allows widget to snapshot into a render-tree, which has OpenGL, Vulkan, and Cairo for software fallback rendering. It also provides diff'ing for minimal damage and all the other important bits.

> It may do that but generally it still uses Cairo.

Uh, no it doesn't. I literally wrote the GL renderer, I know what I'm talking about.

> In any case GTK working on Windows and Mac is an artefact rather than something really usable.

Maybe pre-4, which is why it got redesigned.

> I suspect that efforts to support such multiplatform feature makes Linux native GUI development less developed.

What?

> Generally Linux cannot be considered as a desktop OS as it has no stable and uniform system UI (window manager, graphics, etc.)

Huh? You're out of your mind.

Does Cocoa draw controls like checkboxes and what not with the GPU?

In GTK 4's case, drawing the widgets, CSS transitions, shadows, etc is all on the GPU. Video from GStreamer arrives as GL textures, same for WebKit integration and therefore can be used with all of the other high-level render-tree snapshotting features.

GTK 4 renders with OpenGL, has native a11y built-in, styled with a comprehensive CSS engine, runs on Linux/bsd/Mac/windows, has bindings for almost every noteworthy language…

That issue is literally why I have a branch integrating GLyphy. But yes, your casual drive by comment is definitely more knowledgeable than the people who work on the stuff.

a huge amount of emoji are embedded png in font files. newer ones, however, do require more work.

> figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately.

this is all done in shaping though, the renderers just see "put this thing there" and (at least in GTK) we just maintain texture atlases that are updated on the fly.

you're correct that it doesn't solve the same glyph at lots of scales though. but glyphy does (and at high FPS too).

harfbuzz creator made https://github.com/behdad/glyphy/ which i integrated into GTK a while back (but we don't ship currently, because we still do the bitmap stuff).

the bitmap stuff still has major drawbacks though, like maintaining grid alignments and pixel boundaries where as this stuff (mostly) goes away using glyphy.

still work to be done around hinting though (hence not merged).