6 karma · joined June 29, 2018
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves.
My understanding is the query:
movie
.with(keyword.eq("my-kw"))
.select(title)
is inlined to something by the compiler approximately likefor movie in 0..movie_count {
let lo = keyword_offsets[movie];
let hi = keyword_offsets[movie + 1];
for pos in lo..hi {
let kw = keyword_ids[pos];
if keyword_text[kw] == "my-kw" {
emit(movie, movie_title[movie]);
break;
}
}
}So there's no index lookup on keyword, right? E.g., to use a hash index on keyword to find the resulting movie rows. If I wanted to do so, would I re-normalize the data in some way? This is what surprised me: that even without the secondary indexes, it is still the same (or more) performant than DuckDB.
Perhaps the index-lookup version would use something like
let movies_by_keyword: HashIdx<_, _> =
keyword.text().inv().collect();
but it does not seem like it does (even though I assume DuckDB may).Conceptually, the two key operators `.select()` and `.and()` connect somewhat to the lineage of arrow notation [1], fork algebras [2], etc.
Left-to-right composition:
(>>>) :: (b -> c) -> (c -> d) -> (b -> d)
Fanout:
(&&&) :: (b -> c) -> (b -> d) -> (b -> (c, d))
The database-specific detail is enumerating the universe of entities to maintain shared correlations and using projections as ways to access information, so, in the above notation:
movie :: Movie -> Movie
title :: Movie -> String
movie >>> title :: Movie -> String
The specialization to then have Rust types be able to generate inlined query execution code at compile-time is quite neat. But, I wonder where the limitations of this approach are seen? In contrast to something like Soufflé which has a much more complicated optimizer but generates C++ code ultimately, it seems difficult to always rely on the Rust compiler. Likewise, some other projects like Crepe rely on Rust macros more heavily. And, the embeddability is a plus, but it does not look too easy currently to use from other languages as you would need to compile the query independently and provide something like an FFI wrapper.
Relatedly, some other posts about relational language design from the last couple days: https://news.ycombinator.com/item?id=49342530, https://news.ycombinator.com/item?id=49363617
And this tutorial looks interesting: https://northeastern-datalab.github.io/relational-language-t...
--
[1] https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/arro... [2] https://www.cosc.brocku.ca/Faculty/Winter/JoRMiCS/Vol1/PDF/v...
From `man sandbox-exec`, "The sandbox-exec command is DEPRECATED." However, a lot of AI harnesses seem to use it anyway - I wonder what this says about long-term Apple support.
Oh, and I've found using a tool like gitwatch really helpful for keeping the repo synced without having to remember to commit.