I'm interested in learning about how federated data source query engines like this handle query planning and optimization.
If I wanted to study your implementation for hints where would I start?
I'm interested in learning about how federated data source query engines like this handle query planning and optimization.
If I wanted to study your implementation for hints where would I start?
I'll start by saying that OctoSQL is single-machine-only, as I'm not sure what exactly you meant with "federated".
I'd recommend starting by going with a debugger through the execution of the root function in root.go, as that calls out to all the macro transformations.
Then, you can take a look at the optimizer (optimizer directory) and the Postgres plugin source[0], as an example of a plugin that is able to push down predicates to the underlying database. As well as the Typecheck (logical -> physical) and Materialize (physical -> execution) transformations.
I'm planning to write a few technical documents about the implementation soon, while writing some actual usage documentation as well.