Hacker News Reader

Comment by flakiness | Hacker News Reader

The parser.py [1] has only 1.6k lines. And it is hand-written parser. This size is amazing if it's really capable, but I intuitively doubt it. For example, duckdb's select.y [2] has 3700 lines, and this is only for SELECT. ZetaSQL's grammar file [3] is almost 10k lines.

The SQL is a monstrous language. Is there any trick that keeps the code simple?

[1] https://github.com/tobymao/sqlglot/blob/main/sqlglot/parser.... [2] https://github.com/duckdb/duckdb/blob/master/third_party/lib... [3] https://github.com/google/zetasql/blob/master/zetasql/parser...

View on HN

You should try it out and see. Here are the test cases

https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/...

I can even parse and optimize TPC-H https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/....

SQLGlot's parser handles a superset of SQL, so it's much more flexible than any one engine.

If you look at DuckDB's select.py, you'll notice very "generous" use of newlines, e.g. putting just a ";" on a line, or just "{", or just "}" etc. Partially explains the difference I suppose.

Nitpick: The file is called select.y, not select.py

Was confused seeing these mentions of a select.py and why there would be any noticeable amount of semicolons in a Python file.

Good catch! Fixed.

Maybe they're just good at computers

The code looks good

The parser.py seems extremely terse, not a line of doc strings or typing. Perhaps that explains?

No, SQL isd big an ugly. I wonder if it can do

    select ... (select ...) as some_alias, ...
    from ...

also, can't see SOME/ANY/ALL

SQLGlot supports nested queries and EXISTS. SOME, ANY, and ALL aren't fully supported yet but it's only a couple lines to add support since I already support EXISTS.

https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/... https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/...

I'm impressed.

I've added support now for any, all, and some. Some is identical to Any so I always default to Any.

You can see what it took to add this in this commit.

https://github.com/tobymao/sqlglot/commit/ab49a3a2964bdff3a3...

Reply on news.ycombinator.com