I am not a budget analyst. What bothers me is that I think a field I admire is being mismanaged, not that it is driving the US (or any other country) bankrupt. The US has worse problems than a little bit of science pork.
The big question is: can you detect, a priori, which research projects will be successful?
For example, would you have funded Dave Moon's research at the AI lab? How about L. Peter Deutsch? Ricky Greenblatt? Kernighan and Ritchie? Tony Hoare? Dijkstra? Knuth? James Gosling? Matz?
The irony is that no matter who you consider relevant, you're almost always wrong a decade later. Moon was brilliant at the AI lab, and his work was the foundation for most of the 1980s AI boom. But it's largely irrelevant today. Matz and Guido were relatively unknown throughout the 1990s, but they're suddenly critical now, because technology has shifted towards dynamic languages. Hoare spent most of his life working on arcane topics like proof-carrying code, but think of how many bugs have been found because of assertions.
What if the next big development in computing comes out of type theory? Would you then add Luca Cardelli, Benjamin Pierce, and Phillip Wadler to your "system software designers" category? (Heck, Wadler has already made important contributions to Java 1.5 generics.)
The university system is built upon the premise that you can't tell what basic research will be successful beforehand, and so you better fund all of it (or as much as you can, at least ;-)) and wait for some entrepreneur to sift through it and figure out what's commercially viable. You seem to think that all that random chance is unnecessary, and you can pick the winners beforehand and shut down the rest. If you're right, there's a very lucrative career waiting for you as a venture capitalist, where you get to bet Other People's Money on the future system software designers.
(BTW, I met a girl at a party who's in a swing dancing class with Guy Steele, and he calls himself a Programming Language Designer. So maybe your title for him is a bit off the mark...)
wow! What an insightful post. i regret I can "upmod" you only once! I tried to make the same point in another comment but it is nowhere as clear as this! I am humbled!
You're making my point for me, which is that the process of allocating research funding is driven by nothing other than political power.
The university system is built upon the premise that basic research can justify its funding annually by explaining to Congress the many benefits it will yield to American citizens. There is no "all of it" to fund. An infinite amount of funding will generate an infinite amount of research.
I can tell you exactly why the next big development in computing won't come out of type theory. It's because type theory, at least as it exists presently, is impossible for anyone with an IQ of less than 140 to understand. And it is getting more complex, not less complex.
Isn't a PL designer what I called Steele? Note that he doesn't call himself a PL researcher.
At all the good universities, the computer science department is also part of the electrical engineering department. Thus you have valuable research going on which crosses between hardware, software, information theory, physics, etc.
The people running these departments know that along the way, a few somewhat promising computer scientists will get sucked into reading philosophy books as a hobby, and lose their way in the world. These people become PL researchers (in the sense you're using the phrase) . The school earmarks minimal funds and provides minimal infrastructure, like a USENET group or a mailing list for these types (no pun intended). They know by doing this they will successfully keep them from messing up useful research (best case) or living on the streets (worst case).
But I also think it needs PL design - in the sense of Guy Steele, Paul Graham, Guido von Rossum, etc, etc, etc. And I do think the existence of "PL research" is a serious detriment to the organization and development of PL design. At the very least, they should be parallel subfields.