Hacker News Reader

Apologies for not getting back on this yesterday. Made the mistake of posting from my phone right before evening plans took over.

Looking at this, it seems you build a few indexes, such that I'm guessing those are the main drivers on the benefits? Makes sense, does it add to the space at all? As I said, most teams I work with are still on CSV, so even if this adds, I'm sure it is well below that.

At any rate, thanks for the response. Looks really nice!

View on HN

No worries at all. For the teams which are happy on CSV/JSON, i'd admit that lance is not ideal alternative for them.

> It seems you build a few indexes, such that I'm guessing those are the main drivers on the benefits?

Yeah, we are building different indices into this columnar storage format, which is actually a happy side-effect of its good random-access performance. It does occur extra space for indices.

Thanks for your kind word too!

I'd hazard that many of the teams aren't so much happy with CSV, as they are ignorant of its costs. I fought for a bit to get them to move to parquet, but all too often they insist on having it in a format that excel can open.

Reply on news.ycombinator.com