If the question is "why didn't Unix win?" the answers are basically historical. The original way to create pages dynamically was CGI. The idea was that you'd have some Unix crap on the server you wanted to expose over the web, so you'd write up your form and the form target would be a CGI script. The way CGI is specified, the arguments show up in argv and a bunch of incidental stuff winds up in the environment. This is one reason why PHP's global situation is so strange. Anyway, the request would come in, the server would parse it, fork and exec your CGI binary passing in the data and whatever came out got sent back to the client.
CGI is a great piece of classic Unix engineering, because it's completely language agnostic. Your CGI scripts could be shell, Perl, C, anything. The cost is also classic Unix: it's expensive, due to a lot of forking and re-doing the same work, and it's a bit of a security problem since it raises a lot of questions about which permissions are relevant and who should be running what. These questions led to a couple interesting solutions (suexec and fastCGI) that were apparently too much like epicycles and not enough like ellipses, or we'd all know about them.
mod_php and mod_perl basically killed CGI themselves by solving the performance problem and paring the security problem back down to just the web server itself. I suspect there's heavy inspiration from server-side includes but I wasn't there for that part so I don't know if that's true or just sounds reasonable. The other side of the tree, the Ruby/Python/Java side, wound up deciding that HTTP wasn't so complex it warranted doing a lot of backflips just to go through Apache, so they wound up creating their own servers so they could manage state (both application and interpreter/VM) more appropriately for their platform. This school of thought seems to have won the day, except for PHP and Perl which still tow the hard line for integration.
I have wondered for a while if CGI's performance penalties would really be all that noticeable today, for lightly loaded sites. It would be interesting to take a look into. If so, the problem is really just that one request shouldn't map onto one command. FastCGI was an attempt to create CGIs that endured longer than a single request. My impression is that it didn't become a big deal because by the time it was invented CGI was already on the way out and it's somewhat more brittle.
If there were a reasonable alternate way to represent the request that maps well onto Unix it might be possible to revive the dream.