foldU

This sounds like a really unpleasant shopping experience to me.

The geometric representation of AM/GM is very cool, but the first animation seems wrong to me, it should be varying the value of `b`, not the location of the circle, for it to make sense, no?

I don’t mean to suggest it’s not sleazy—I think it’s sleazy by default. VC writing is not to be trusted.

I don't think a post on a VC blog of all places should be considered "journalism"

This is correct, I appreciate you for putting it so coherently :). I think I didn’t make it clear enough in the piece that I’m coming from a stance of fast access being table stakes, and the question being about how that’s accomplished.

he addresses this, you simply get airline pilots to pick them

Yes, it’s good

whoops! thank you!

(author of OP) That post of yours was actually what got me tooling around with this stuff again :) it's a really excellent one

You also don't really have to worry about convergence, since these are formal power series.

More deserves to be written on the ILP idea, I haven't actually tried to make it work but it seems to me like the only real direction to optimize ("optimize" used in the mathematical sense, rather than the programming sense) queries that you can't just exploit the principle of optimality on (see this earlier article I wrote [1] for a bit more exposition). I think maybe some of the egg [2] people have experimented with this.

Speaking from experience, there's lots of rewrites you'd want to do in a query optimizer that having access to efficient DAG-shaped query plans would make tenable, but as a specific example they are an important part of doing full subquery decorrelation [3].

[1] https://buttondown.email/jaffray/archive/why-are-query-plans...

[2] https://egraphs-good.github.io/

[3] https://www.scattered-thoughts.net/writing/materialize-decor...

Ah my apologies, I wasn't familiar with that syntax.

I think this one is obvious actually! There's no choice but to aggregate it at the level of the `xx` `SELECT`, no other level has access to the `x` column.

(I'm the author of the original post [1]) And while I think the rules, in the end, make sense, I think it's not quite as clear cut as you describe, consider:

    SELECT (SELECT sum(1) FROM xx LIMIT 1) FROM aa;

which returns

    3
    3
    3

Personally, I think having the inner aggregation always attach to the nearest `SELECT` would have been an equally valid way of defining how this works, but it just so happens it is not defined that way.

[1]: https://buttondown.email/jaffray/archive/sql-scoping-is-surp...

I agree, I think the original sin here is the fact that whether a `SELECT` is an aggregation is determined by the contents of the scalar expressions at all. I think most of this weirdness comes directly out of wanting to be able to write both `SELECT sum(x) FROM xx` and `SELECT x FROM xx` and have them work.

Not that I have a better solution offhand, in SQL grouping by a constant value is not actually the same as not writing `GROUP BY` at all since the behaviour on empty tables is different.

I had not heard of this term but it sounds like the same thing!

whoops embarrassing. thank you for pointing that out!

I had no idea, thanks!

You are correct! I will fix it when I get home, thank you for the correction!

This implementation incidentally uses a hash table, but any set-like data structure would work. This is different from things like HyperLogLog that actually depend on the hash values themselves in order to work.

Thanks so much for the thoughtful response! TigerBeetle seems very cool, I hope to dig in and learn more about it at some point. As an aside, your Twitter thread on macOS fsync behaviour was very helpful for me!

Yes my intention was that it's more or less the reference version of that model, certainly databases like System R existed many years before Volcano :) I appreciate the correction.

Both the linked papers (Neumann[0] and Shaikhha et al[1]) go into this a bit. I personally found figures 3 and 4 in the Neumann paper particularly helpful.

[0] https://www.vldb.org/pvldb/vol4/p539-neumann.pdf

[1] https://arxiv.org/pdf/1610.09166.pdf

(I'm the author of this post) That's a great document I hadn't seen, thanks for sharing!

Agreed! Peter has actually done some really great research into this, finding real vulnerabilities in applications due to insufficient transaction isolation: http://www.bailis.org/papers/acidrain-sigmod2017.pdf

Sorry, that was my fault! Should be fixed now.

Good question! In 2.0 we only use one of the fields for the index lookup and then use the rest as a filter. For 2.1 we’re planning on augmenting our execution engine to enable making use of all the fields in a more efficient way. We have an RFC[0] describing the way our inverted indexes are designed, though not everything outlined in that document is implemented yet.

[0]: https://github.com/cockroachdb/cockroach/blob/master/docs/RF...

Thank you for the nice comment, and for the lovely post on Prolog meta-interpreters that I linked here :) It seems I missed the part where you explicitly called out absorption and reification there! One of my reviewers actually suggested "elision" as well as reification after I had published the post, which seems quite similar.

And thanks for the info! I would love to learn more about compiler design and garbage collection in the future. Still working my way through some other reading materials but certainly something I would like to get to more!