Also, very often, it's not software that does not scale, it's people working on the software. Clearly, there are relatively few developers in the marketplace who can master the most advanced software development techniques. In practice, "clever" code will be misunderstood sooner or later, its complexity will increase as "lesser" developers patch it down to a level where they can understand it, expand it and correct it. And so you end up with large balls of mud, initially clever code turned into tremendous sources of complexity.
If you look at the pillars of Internet, HTTP, FTP, telnet, IP, TCP, POP, SMTP, etc. There is 1 common factor between them: They are dead simple solutions to specific problems. And that is why they stood the test of time.