If I never use multiple languages in the same process can I safely ignore it or are there other benefits?
If I never use multiple languages in the same process can I safely ignore it or are there other benefits?
On the nodejs services + frontend JS side, there was no real equiv tool for interop -- traditional soln is slowly round-tripping through SQL/ORM or manually doing protobuf through some sort of pubsub -- so Arrow is part of how we have been taming that mess too. It'll be a longer journey for Arrow or something like it to get adopted in JS land, but I can see folks doing TypeScript and serverless wanting it for a cleaner solution to typed data, faster serialization & streaming, etc. (There is no true JS equiv of pandas nor a typed variant.) We were a bit early here b/c we wanted streaming through WebGL + OpenCL/CUDA, and while we are fans of typed data, found protobuf tooling to be too unintegrated and manual in practice.
As you mention in your second paragraph, arrow is perfect for getting typed query results from a database server directly to a web frontend with minimal overhead. With the Transferable interface, it's even possible to zero-copy transfer the arrow data from the network buffer to a webworker.
edit - this looks a good explanation, parquet is more widely used and smaller, but slower. https://stackoverflow.com/questions/48083405/what-are-the-di...