EuclidesDB: a multi-model machine learning feature database
euclidesdb.readthedocs.io
euclidesdb.readthedocs.io
There's a big market of people who are looking to do simple data science and machine learning but don't know how to get started, don't have a lot of expertise to implement the algorithms, and the required ETL looks really daunting. You could put all of this on rails by looking integrating with existing systems.
If I'm searching for a particular breed of dog, and let's say EuclidesDB has already been trained. I just need to query based on my image of the dog breed?
How does it differ from me interacting directly with a model already saved after training on Tensorflow?
Im not familiar with EuclidesDB, but from what i've understand from the documented architecture it uses gRPC to expose a RPC interface over the internet, and abstract away the objects in the domain of machine learning (like images) tied to PyTorch.
Anyway, its a higher level concept and architecture modeled on top of a key-value storage like LMDB, which is a lower level building block for this product. LMDB being comparable to LevelDB, not EuclidesDB.
(It looks that the author is here, so please correct me if im mistaken in any way)
Any reason why you chose LevelDB?
I was actually looking at clipper.ai for model serving at my work. I know it's not 1-to-1 comparison as clipper is much more generic where as this seem to tie in closely with PyTorch. Can it support models created using other libraries?
Of course if your new domain is very different than the distribution of the original training data, it is a good technique to fine-tune the network a bit with your data.
Model A = fine-tuned to classify between different types of shoes;
Model B = fine-tuned to classify between different t-shirt types;
EuclidesDB can have these two models and you can add/query items into each one of these different models (hence the concept of "model/module space" that is used by EuclidesDB).
* CNN (plenty of pre-trained * Approximate Nearest Neighbors Database (Annoy, etc) * webserver to host CNN and serve UI
It's popular to serve tensorflow models w/ tensorflow severing + kubernetes and that's what' I've done in the past.
This coupling to pytorch is very cool, but basing this on a production capable database like postgres (which has incredible hosted solutions like Google Cloud SQL, AWS RDS, Azure , etc) would be much more useful.
Those interested can also find an open source integration of lmdb + annoy here: https://github.com/jolibrain/deepdetect/blob/master/src/sims...
This the underlying support for similarity search based on embeddings, including images and object similarity search, see https://github.com/jolibrain/deepdetect/tree/master/demo/obj...
This is running for apps such as a Shazam for art, faster annotation tooling and text similarity search.
Annoy only supports indexing once, while hnwlib supports incremental indexing, something I'm looking at.
There are many reasons why we depart from other libraries, many of them, for instance, uses JSON+base64 (http/1) for serialization, while we use protobuf+gGRPC (http/2).
R and Python (pickle, for instance) both have facilities for serializing objects that can (sometimes) be used for ML models. Additionally, tensorflow has an API for saving models and I've seen people save the coefficients and structure of neural nets in HDF5 files (https://danieltakeshi.github.io/2017/07/06/saving-neural-net...).
But I wouldn't say there is a cross platform, universally adopted, model agnostic standard at this point (and it would be very difficult to create such a thing).
It's tempting to think that everyone else has this figured out but I don't think so.
Our model server just works with every format. We created our own DL framework and maintain that. Eg: We're the 2nd largest contributor to keras alongside having our own DL framework.
We've found it's easier to just have 1 runtime with PMML, TF file import, keras file import, onnx interop coupled with endpoints for reading and writing numpy, apache arrow's tensor format, and our own libraries binary format. We also have converters for r and python pickle to pmml as well. Happy to elaborate if interested.
For ingest we typically allow storage of transformations, accepting raw images, arrays etc. We have our own HDF5 bindings as well but mainly for reading keras files. Could you tell me a bit about what you might want for ETL from HDf5?
Would you be willing to characterise the sort of system that you end up replacing, what your customers were doing before using your service?
An interesting example was a customer that bought a startup. The startup had a docker container that leaked memory (they were using an advanced TF library, google quit maintaining it). The parent company had tomcat infrastructure. They told the startup to get it to work in tomcat. They approached us.
Dl4j itself is built on top of a more fundamental building block that allows us to do some cool stuff in c: https://github.com/bytedeco/javacpp-presets
We maintain the above as well. We have our own tensorflow, caffe, opencv,onnx etc bindings. This gives you the same kind of pointer/zero copy interop you would in python but in java.
Another area we play in is RPA. You have windows consultants who don't know machine learning but deal with business process data all day. They want standardized APIs and to run on prem on windows: https://www.uipath.com/webinars/rpa-innovation-week/custom-b...
They want the ability to automatically maintain models after they are implemented with an integration that auto corrects the model. We do that using our platform's apis. On line retraining is another thing we do.
We also do automatically rollback based on feedback if you specify a test set. If we find the test is less accurate for a specified metric over time, we roll back the model. This prevents a problem in machine learning called concept drift which means the domain changes over time.
Lastly, you have customers generating their own models and you need to track/audit them all. Sometimes to do online retraining like the RPA use case. Some customers scale to millions of models. They need this kind of tracking.