Doesn't tolerance work both ways? One brick is 1um longer than it should, another one is 1um shorter - in the row of 100 bricks those differences will mostly cancel each other out, won't they?
Also, it's not just about variance between different bricks (1x1 vs 2x1) it's about variance between different parts out of the same tool (the correct term for a mould is a "tool"), and variance between different tools producing the same brick.
Some really subtle effects affect the size of parts if you care down to the level of 1um - all sorts of temperature variation, pressure, humidity, colour mix etc.. Lego's skill is the ability to control or compensate for all of these confounding factors and get consistent results, regardless of age of mould tool, manufacturer of machine, ambient conditions etc.