For those of us with slow internet connections please change your Makefile to use a shallow clone:
``` github/three: mkdir -p github git clone --depth 1 https://github.com/mrdoob/three.js.git github/three -b r108 cd github/three && git checkout r108 ```
This would save a couple of gigs and hours in download.
Just skimming your code I see variable mangling but no AST optimizations, is that correct? Which is odd because you show similar output sizes to the JS bundlers.
You probably aren't seeing the AST optimizations because they are actually in the parser. Search for "mangleSyntax" to find the optimization code.
This would normally be done in a separate pass but I'm trying to minimize the number of passes done over the full AST for speed. I figured it'd be faster to optimize the AST as it's parsed because the newly-created AST nodes are likely to still be in the CPU cache. That said, I haven't tested the performance impact of doing this. I just did this based on intuitions from performance work on previous compiler projects.
By the way, it doesn't affect timings by more than 20%, but that repo has locked in rather old versions of rollup, terser, and other packages. You can disable tree shaking in rollup with --no-treeshake and still get comparable bundle sizes after using terser. That would save 3 seconds. There's no tree shaking opportunity in re-exporting all the symbols from the same library 10 times - all code must be retained. Using terser({compress: false}) would save an additional 17 seconds at the expense of making the final bundle 3% larger - 6.01mb total. Mind you even with upgraded packages and these config tweaks rollup+terser would still be 30x slower than esbuild - compiled languages will always win in speed.
Bundling and minifying a real app rather than a library might yield more interesting bundle size numbers due to dead code elimination. Providing gzip sizes would be interesting too - they aren't always proportional to the original ungzipped sizes.
It could definitely be useful to have additional benchmarks that test other configurations though. For example, development build speed is also relevant and would make a good benchmark. I'll think about adding more benchmarks if I have time.
I also want a benchmark that tests a real app. I just haven't yet found a large-enough open source app to use as a benchmark. Do you know of one? I work at Figma and Figma's main app makes a great test case, but obviously using that as benchmark wouldn't be reproducible by others outside of Figma. Figma's code base is around the same size as the benchmark I'm using (so 10x the size of three.js).
Ultimately I'm planning on implementing a TypeScript parser in esbuild, at which point I'd like to switch to a benchmark of a TypeScript code base. I also want to include multiple entry points in the benchmark because that's something Figma's main app uses, and esbuild has specific optimizations for that case.
Long story short the benchmark I'm currently using was helpful for me during the initial development but it's not the final one I plan to use, and I will likely completely change it as the project evolves.
Edit: Oh yeah and gzip sizes are great to track. I've tracked these for previous compiler projects I've done and you're right, sometimes you need to make the output larger to make the gzip output smaller. I haven't looked at the gzip sizes at all yet and there is room for improvement. Check out https://github.com/evanw/esbuild/issues/6#issuecomment-58667... for an idea of what specific numbers might currently look like for esbuild compared to other minifiers.
Unfortunately I am not aware of a large public ES codebase to test against. Most projects use Typescript as you mentioned. You could grab the ES bundle made by several projects and test against that, but that also wouldn't be a fair comparison.
The results in https://user-images.githubusercontent.com/406394/74600139-d1... are very informative. It matches my expectations.
I've been studying the Go code in esbuild - very clean and extensible. I think this project has a lot of potential. Thanks for sharing it with the world.