The speedup from tools like esbuild, swc, etc. are not because of the language they were written in but because they're competing against Babel. Babel is this single-threaded highly-extensible behemoth. On top of this, you're encouraged to install dozens of plugins, and each plugin has access to global state.
The new transpilers have worked around this problem by introducing various limitations. esbuild plugins literally can't modify the AST. It's basically a way for you to run a command on a string/file. So sure, if you redefine the problem as string transforms rather than AST transforms, you can get an O(N) speedup...