It's really tiresome.
SQL is not a particularly well thought-out language Uhm, that not a very substantiative argument - care to clarify? Bear in mind that the structure of the language is 100% dictated by the Relational Algebra - form follows function to the letter. And the very constraints that algebra puts on what you can do are the same constraints that allow the system to function based on your intent, so that you don't need to define or in most cases even chose the algorithms. The only wiggle room you have in designing a language based on relational algebra is the choice of tokens and ordering of language constructs (e.g. FROM before SELECT of after). So?
But even if it were, the syntax is still poorly thought out. There's no consistency in SQL; each command has its own grammar. You "SELECT ... FROM" and then "INSERT INTO ...". Command syntax is effectively arranged at random.
Worse, there's no extensibility. Most programming languages, when faced with the problem of manipulating data, make use of functions. Instead of having specific syntax for selecting columns or filtering by a condition, a more sensible approach would be to make them functions that lazily transform tables. Then the user could define their own operations, and we'd no longer need to generate a page of SQL each time we wanted to do something complex.
Next problem; integration with calling languages. SQL is not designed to integrate well with ORMs or other client libraries. We either want a language that's basically little more than a serialized AST, or take an approach similar to CouchDB, and farm out specific calculations to some server or library running our programming language of choice.
Finally; optimization opaqueness. It can often be difficult to predict how the database server will optimize a particular piece of SQL. It might be useful if we could drop down to more primitive, but more predictable, operations when necessary.
In short, I think that if a relational database system were to be designed from scratch by experienced programmers, it would look very different to the SQL databases that are around today.
I spent a lot of time writing SQL and the ordering of SELECT ... FROM doesn't bother me at all. Frankly, I find this argument silly and I do not want to entertain it - when buying offroad SUV I will only compare the color of door handles when all else is equal, which it isn't. The only serious part of it is intellisense problems which admittedly could use some work (this is why LINQ is the other way around - table name is always spelled first).
The other two points are entirely valid. The integration is a real pain. The opaqueness can be mitigated by vendors as well - MSSSQL allows hints to force desired behavior - but that doesn't so much drop down in abstraction level as it bastardizes the existing one.
users where(u => u.login = "foo" and u.passwd = SHA1("secret" + u.salt))
authenticate(login, password) :=
where(u => u.login = login and u.passwd = SHA1(password + u.salt))
users authenticate("foo", "secret")
Because SQL is so limited in what you can abstract, there's very little room to factor out common code. Views, stored procedures and even T-SQL functions just operator on set tables. With a more flexible syntax, one could have third-party libraries containing more complex table transformations.
Personally, I wonder whether we need a separate language for database queries at all. It would be nice if there were a just a library we could use to create our own database server inline, using whatever language we wish.
My guess as to why it isn't possible is that in the database world there is an assumption that all records of the same type lie in one table, i.e. the table is the type. And if you look at it form theoretical relational theory point of view, that is so. Alas in practice we do have things like third-party components and we have to worry about lifetime (e.g. transient table-typed variables).
To be fair Oracle I think has something called "package" which allows writing reusable code. Don't know much about it, but supposedly that's what it does.
Also, MSSQL has notion of Table Type which can be used to declare variables. I actually implemented (a part of) it while I worked there. :)
See, this goes back to my point - people aren't learning their tools and then complain about perceived lack of functionality.
I suggest hitting stackoverflow with this type of questions in the future. That's what I always do when I wonder "can I do X with Y?".
The reason why I don't like SQL much, is not because of SQL itself, but because of its poor fit into the host programming language.
In PL-SQL, Oracle's own programming language, the SQL statements are simply part of the program. That is how it should be.
But in other languages (like Perl), the SQL is like code you need to `eval` at runtime. There is no syntax check until you actually run it. And that makes it cumbersome to work with. Plus, you often have to do optimizations (like to separate `prepare` from the rest of the code) by hand, which only makes it worse.
It's all just very low level.
I respectfully disagree.