Hacker News Reader

Comment by wcrichton | Hacker News Reader

This is essentially what an ORM does, e.g. SQLAlchemy [1]. Except instead of providing a slightly nicer API to generate query strings, an ORM tends to be more tied to the host language and use its object system to abstract out most of the SQL complexity and make declaring SQL tables/queries look like normal operations on the data themselves instead of the database. Django's ORM [2] for example autogenerates joins for you under certain query conditions, which is super handy.

[1] http://docs.sqlalchemy.org/en/latest/orm/tutorial.html

[2] https://docs.djangoproject.com/en/1.11/topics/db/queries/

View on HN

I'm aware of SQLAlchemy Core/ORM (and ORMs in general), my question is more like why do we have to concatenate strings in these project's internals instead of using some API to pass query structures without intermediate composing/decomposing. For small queries that ORMs often do, it feels like needless and annoying overhead.

For now, even '?' placeholder escaping is actually manually-printf'ed under the driver's hood, afaik, so in the end we send plain text to the server. SQLite probably avoids that via sqlite3_bind() call, which allows to pass SQLITE3_STATIC string args if you want, but that's the benefit of in-process engine.

I highly doubt that placeholders are interpolated into the SQL statement. Doing so will defeat the purpose of having them in the first place. Rather, they are tranferred out-of-band to the SQL "engine", as it were, and are used as straight up variables.

Depends. Server-side prepare works the way you described while client-side prepare interpolates placeholder values. Both are widely used.

I checked it for PostgreSQL right now, and it seems that libpq can send them OOB via PQexecParams() and networked integer buffers. But that's not the case with OpenResty's pgmoon driver, which doesn't have such an interface. E.g. Lapis framework built on top of that simply escape_literal()-izes all arguments and interpolates right into query text. So, YMMW with specific (or generic) drivers.

Placeholders are there for those who tries to concatenate queries by hand; having them properly escaped in protocol should not defeat client-side security purpose.

You can never properly escape user input, so this escape_literal() is a ticking timebomb.

How come it's impossible to properly escape user input? I get that it's hard if the format is complex, but I don't see what makes it categorically impossible, especially for a simple format like an SQL string. You just double every single quote and then surround the whole thing with single quotes.

> ...every single quote...

And then you come across \u0027 and you're screwed.

If you use a broken RDBMS, maybe. The standard is quite clear, and that certainly doesn't work in PortgreSQL.

Do you know any decent server that interprets 6-char \uxxxx sequences as part of its top-level query syntax? I mean, how do you safely pass '\u0027' literal to it anyway? If you can't, then its literals and escaping are broken by design, so it is good to know from the start of using it.

Edit: removed double backslash from literal to not confuse it with host language's escaping.

Having the interpolation done in one central place and coded by someone experienced sounds as good as OOB to me.

With that central place being?

Lapis db:query("? ?", a, b)

Edit: this call is most low-level db interface in lapis apps.

See the part about support-costs in a different link.

But even more, I think it's "in the nature of programming" that for a given system you have some parts are regularly structured and externally visible but other parts that seem like they ought to be visible and changable from outside but which aren't and can't easily be made so.

This might be because they use intermediate variable or it might be because they have changed the implementation over time or for a variety of reasons.

And this ultimately comes down to the difference between the way intuitively you'd think X feature should be implemented and real, much-more-messy way X feature is actually implemented - the difference internals and an API.

Having internals and externals be equally easy to work with was the dream of the object orientation but sadly that has remained a fantasy and unfortunately looks like it will remain so.

You need some way of serializing your data structures to a wire format, and SQL has worked ok so far.

Besides a potential of a performance gain in parsing at the server side, I'm not quite sure there's much gain you would get - there might not be that much difference in serializing those data structures to "plain text" vs any other format you come up with.

Reply on news.ycombinator.com