Carmack, as dictator of id code (and thus id design) had the ability to say "we are not adding the following feature to our game because I don't like the compromises it will force in our technology and, thus, in other features we want to add down the road." It's basically as simple as that.
In almost all other organizations, someone holding the pot of money will trump the programming overlord at that point and say, "I don't care, XML is the buzzword this second, and all objects in the game need to be serialized to XML as web services - that's what the market is clamoring for right now" (or whatever the buzzword du jour is). And over time those political technical decisions add up to so much cruft that adding any new thing becomes a technical, jenga-style nightmare.
Not so with id's engines. I actually worked on beta versions of both Quake 2 and Doom 3 code bases while those games were in development, and one striking thing about getting updates from Carmack was his ruthless willingness to eliminate working, finished features / code if he didn't think the complexity or dependencies they introduced were worth whatever feature they added in the overall technical context.
Quake is what you get when a genius programmer who has mastery of his knowledge domains leads. Institutionally, there aren't too many other circumstances where similarly talented and sure-footed and, well, fundamentally, _right_ programmers get to specify requirements like that. I think the comparison to the Linux kernal is apt.