Hacker News Reader

hawaiianSporkon Succinct data structures

How do succinct data structures do with vector operations on cpu?

Not sure if they are succinct, but the Apache arrow format encodes data in several ways that is compact in memory but also allows operations on these structures.

hawaiianSporkon Understanding Parquet, Iceberg and Data Lakehouses

Parquet has been the lakehouse file format of choice for nearly half a decade. But we are starting to see other contenders that are optimized more for lower latency like lance https://github.com/lancedb/lance

hawaiianSporkon The Graph Programming Language GP (2009) [pdf]

I looked for the York Abstract Machine used in the paper and couldn't find anything about it outside of this paper: https://www.cs.york.ac.uk/plasma/publications/pdf/ManningPlu...

It would be nice to play with the code.

hawaiianSporkon Zetk: CLI utilities, fzf-based scripts, C++ lib for managing your Zettelkasten

I wonder if exposing this as a language server would be helpful?

hawaiianSporkon Cue, an open-source data validation language

If you are looking to do data validation from the JVM, you may try Baleen (written in Kotlin): https://github.com/ShopRunner/baleen/

I'm one of the contributors. We created a DSL in the language to describe the data and create tests. You can then use that data description to validate against json, csv, avro... One of the neat things we came up with was the concept of a data trace which is like a stack trace but is a path through the data to a particular error.