Open Sourcing Venice – LinkedIn’s Derived Data Platform
engineering.linkedin.com
engineering.linkedin.com
We don’t know of any other system (at least open-sourced) that solves this problem at our scale. In the conference where we announced Venice, we learnt about some hacks that other organizations have been employing to address this problem in traditional data stores.
I am actually working on a side project several years on now that may benefit from Venice at some point :) Thanks again!
As per decreasing the cost of development over the next few years, it is actually quite the opposite. In the near term, the cost of development increases as the system not-only needs to work in your organization, but also work externally - with external technologies.
You are right that reducing the cost of development is a goal, but there are many things that need to be done initially to facilitate that and make the project self-sustaining.