Hacker News Reader

Wait so you're telling me I could store data as a PDF file, and access it easily / quickly as SQL?

If you found/wrote a adapter to translate your structured PDF into Arrow's format, yes - the idea is that you can wire up anything that can produce Arrow data to anything that can consume Arrow data.

I'm kinda confused. Is that not the case for literally everything? "You can send me data of format X, all I ask is that you be able to produce format X" ?

I'm assuming that I'm missing something fwiw, not trying to diminish the value.

The difference is that arrow’s mapping behind the scenes enables automatic translation to any implemented “plugin” that is on the user’s implementation of arrow. You can extend arrows format to make it automatically map to whatever you want, basically.

And it’s all stored in memory - so much faster access to complex data relationships than anything that exists to my knowledge.

Could I write a plug-in that mapped to Cypher? I’ve got a graph use case in mind where I want to use RedisGraph but don’t feel comfy with Redis as a primary DB and would totally consider a columnar store as a primary if I didn’t have to serialize.

Reply on news.ycombinator.com