V8 doesn't use an AST, it uses a CFG, but I believe it comes down to efficiency - it's far cheaper to look at the length of a string than to traverse a graph, and by its nature JS needs very fast compilation times.
This is probably one of those heuristics that works well enough on enough real world code, even though everyone knows it's suboptimal. I've heard that the turbofan compiler will remove this limitation but that's still very much work in progress.