It also gets extraordinarily complex when you have to handle text layout, kerning, languages that read in different directions, styles, and all of the other complications that come with font rendering.
TrueType fonts even support a small virtual machine to process tiny code programs related to font rendering. It's an incredibly deep topic.
Or even WebAssembly (though more of an experiment than anything): https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-sha...
Performance optimizations mentioned in the other comments is not the sole reason. Another one is visual quality.
Direct rendering of these vector glyphs would look good on high DPI displays and/or for very large font sizes. For normal-looking text on typical displays you won’t achieve great results without hinting: https://en.wikipedia.org/wiki/Font_hinting
Another quality related problem is subpixel rendering https://en.wikipedia.org/wiki/Subpixel_rendering Just like hinting, hard to solve while directly rendering vector shapes.