Hacker News Reader

Comment by choeger | Hacker News Reader

Maybe a stupid question, since I am not a web dev, but how exactly do you render the text with the GPU? Text rendering ain't simple. On top of that, how does that interact with the browser? Can the browser still select the font (size) of the rendered text? Can the user select it for copying?

View on HN

For the latter questions, WebGL is rendered as an opaque bitmap in the browser (canvas element) so it amounts to an unselectable image as far as the user is concerned: It's all pixels and no text.

It’d be a fair bit of work, but from a technical point of view you could implement a click and a drag event handler that “selects” the text, and intercept Ctrl+C etc and place the text on the browser clipboard.

Ultimately it would probably feel quite unnatural compared to the native text selection of the browser. And probably would not work very well except in a small subset of browsers.

I think at some point in the future browsers and/or operating systems will ship with machine learning powered systems for recognizing text in bitmaps and for letting users copy recognized text. This is already possible today both with third party non-ML powered OCR software and with third party ML-powered software.

The new Google docs uses a canvas. The feature is not rolled out fully yet but there is a demo document attached to the announcement. The canvas demo document is fully selectable, it feels native. I assume it works as you described, so it's possible to make it feel right, with enough effort to detail.

By placing the right elements at the right place, you can have native input and text selection, but that’s not just WebGL anymore; That’s a regular (but transparent) webpage with a WebGL background that mimics your input. It takes quite a bit of effort to emulate that effectively since there’s no guarantee that the visible text rendered by WebGL renders with what the browser renders outside it. Probably the 80/20 rule applies here.

I inspected the DOM while selecting and I didn't find the hidden text element. Perhaps it was there and I didn't know how to find it.

Is this a more likely approach than just drawing the selection highlight and emulating the whole thing in canvas?

> how exactly do you render the text with the GPU?

This library uses a prebaked SDF texture.

https://github.com/wwwtyro/candygraph/blob/master/src/assets...

> On top of that, how does that interact with the browser? Can the browser still select the font (size) of the rendered text? Can the user select it for copying?

No.

Reply on news.ycombinator.com