I checked how long linking takes for quick-lint-js [1] with Mold. 108 milliseconds. That's pretty chunky. I should try .so files like you suggest.

[1] the full project, not the 17k SLOC subset discussed in the article