Hacker News Reader

Comment by berkes | Hacker News Reader

I'm curious about that too.

From my superficial knowledge of compilers, "modularization" itself should not make code faster, if anything slower. There'll always be some overhead of loading modules and communicating between them, not?

I presume, from my own experience when building software (not compilers), that modules allow for a much easier to reason about, much better isolated (cohesion, loose coupling). And therefore for much easier improvements inside the module. I would presume that, here too, modules allowed them to improve the inner workings much better, allowing for the performance increase. Or am I completely misunderstanding this feature?

View on HN

There are some key things here that maybe weren't clearly stated in my writeup.

Firstly, the old codebase is TS namespaces, which compile down to IIFEs that push properties onto objects. Each file that declares that namespace is its own IIFE, and so every access to other files incurs the overhead of a property access.

With modules, tooling like esbuild, rollup, can now actually see those dependencies (now they are standard ES module imports) and optimize access to them. In this PR's case, the main boost comes from scope hoisting.

For example, in one file, we may declare the helper `isIdentifier`. In namespaces, we would write `isIdentifier` in another file, but this would at emit time turn into `ts.isIdentifier`, which is slower. Now, we import that helper, and then esbuild (or rollup) can see that exact symbol. All of the helpers get pulled to the top of the output bundle, and calls to those helpers are direct.

That's why modules gives us a boost. There's also more (modules means we can use tooling to tree shake the output, and smaller bundles are faster to load), but the hoisting is the big thing.

Maybe they could update esbuild to be aware of TS namespaces instead?

I think it's the case that modules are the future; but the main focus of this change was not actually performance at all. We've been wanting to be able to dogfood the modules experience (used by most TS devs) for a long time. The fact that it turns out to be so much faster is a really great side effect.

TS namespaces are a really interesting relic of Typescript pre-1.0 in the bad old "jQuery era" before ECMAScript Modules and even before Node CommonJS modules were that dominant and the two most common "module" formats in the browser were "no module at all" and less beloved AMD [RIP]. Typescript namespaces were based on one of the IIFE approaches to "no module at all" smashing a sometimes large codebase into a single global variable, jQuery style.

Most Typescript projects today wouldn't use TS namespaces if you paid them too. It's a backwards "module format" that the modern web and modern Node (and Deno) is trying to leave behind. Several issues and PRs have been filed on Typescript to drop namespaces as a first-class syntax altogether because it unnecessarily confuses newcomers and shouldn't be used in new code in 2022, but there are some major pre-1.0 Typescript projects that still need them for legacy reasons. Typescript itself bootstrapped itself with itself and was one of those large projects with such a legacy dependency, hah. (From the PR you can see precisely how much tech debt that this has left in Typescript's own codebase!)

So, long story short: esbuild doing a bunch of work to support jQuery-era IIFE patterns is maybe not the best use of esbuild developers' time in 2022.

There are two performance implications of "modularization": initialization-time and run-time.

You are correct that initializing many modules is usually slower than initializing one module [1]. However, bundling puts all modules into one file, so this PR doesn't actually change anything here. Both before and after this PR, the TypeScript compiler will be published as a single file.

At run-time, switching to ES modules from another JavaScript module system can be a significant performance improvement because it removes the overhead of communicating between them. Other module systems (e.g. TypeScript namespaces, CommonJS modules) use dynamic property accesses to reference identifiers in other modules while ES modules use static binding to reference the identifiers in other modules directly. Dynamic property access can be a big performance penalty in a large code base. Here's an example of the performance improvement that switching to ES modules alone can bring: https://github.com/microsoft/TypeScript/issues/39247.

[1] This is almost always true. A random exception to this is that some buggy compilers have O(n^2) behavior with respect to the number of certain kinds of symbols in a scope, so having too many of those symbols in a single scope can get really slow (and thus splitting your code into separate modules may actually improve initialization time). This issue is most severe in old versions of JavaScriptCore: https://github.com/evanw/esbuild/issues/478. When bundling, esbuild deliberately modifies the code to avoid the JavaScript features that cause this behavior.

> From my superficial knowledge of compilers, "modularization" itself should not make code faster, if anything slower. There'll always be some overhead of loading modules and communicating between them, not?

I think this is a misunderstanding of what actually happened.

TypeScript has a thing called “namespaces” and a thing called “modules”. Both provide modularization. The TS repo is not being modularized, instead, the namespaces are getting converted to modules.

Namespaces are an old-school approach to writing a module in JavaScript. You pack all of your exports into a JS object, and then access the object from somewhere else. This works, but JS is dynamic, and the runtime has no way to guarantee that you won’t mess with this object (replace functions or whatnot).

Modules don’t have this object. You just call the function, instantiate the class, or do whatever else with the names you imported. They are resolved statically, so certain optimizations become more “obvious”, like inlining.

For ES6 modules, the exports object is frozen (made read-only) so the JIT can make some extra assumptions and optimizations. With bundles, unless the bundler inserts `Object.freeze` around `module.exports`, they have to be treated as dynamic objects.

Reply on news.ycombinator.com