HNHacker News
TopNewBestAskShowJobs

dmpetrov

193 karma · joined February 22, 2015

AI dev-tools. Former Data Scientist @ MSFT
submissionscomments
dmpetrov··on Show HN: DVC Studio – Git-Based ML Experiments Management
Hey everyone, our team is working on open-source tools for data scientists: dvc.org and cml.dev. These two products help ML teams track ML experiments and run training in the cloud using Git & GitOps approach.

Today we are launching DVC Studio - User Interface for DVC and CML. This UI works on top of GitLab, GitHub or BitBucket and extends it by ML specific scenarios:

- Visualizing dashboard of ML experiments

- Graphs for your ML training

- Manages connections to your clouds - data is not stored in Git, but cloud storages :)

- Modify hyperparameters in UI & run ML experiments in clouds or Kubernetes

All of this through Git, GitOps paradigm and with connection to GitLab, GitHub and BitBucket.

Looking forward to your feedback!

dmpetrov··on Why do most top performers have the highest count of commits and pull requests?
In a mature project, the number of removed lines is a stronger proxy to productivities.

It is related to the unavoidable tech debt that any project has and only the strongest people see it and work on it.

dmpetrov··on Ask HN: How to Start Contributing?
You can search for Hacktoberfest to find some relevant materials.

PS: it was an accident this year related to hacktoberfest :) but it is still good starting point

dmpetrov··on The Multi-Path Career
> There are so many early-growth startups that could really use experienced leaders but don't have the funding and/or the track record to attract the kind of people they need on a full-time basis.

I mostly know early-stage companies (10-300 people range). It is not easy to imagine how it might work on this scale. I'd appreciate some examples (job postings/blog posts).

dmpetrov··on The Multi-Path Career
> it will become the ideal work-style for executives

I personally don't believe in this. The executive role is already multifunctional - you need to were multiple hats on the same day or even at the same meeting.

The biggest problem most of the executives have is time. It is a problem even for a single company. I don't think any company is interested in a part-time CEO/CTO. Some exceptions are possible (Jack Dorsey, interim-CEOs) but no more than that.

However, for ICs this trend will happen. People are not limited by a location and can do Consulting type job in any location. It is a great opportunity for businesses as well - to hire part-time the best of the best.

dmpetrov··on Machine Learning Engineer Guide: Feature Store vs. Data Warehouse
That's might be fine. Anyway, you need to retrain your models quite often due to the concept drift. Solving data problems like this is just a part of model retraining (and concept drift).
dmpetrov··on Show HN: My #1 game as a nontechie girl- Word Hookup -hook letters to make words
It looks like a cool game. A short video on the page might be helpful (like the one in the blog post).
dmpetrov··on Ask HN: What “seeds” can I plant early and reap the benefits later?
"learn to code" Paul Graham
dmpetrov··on Open-Source Version Control System for Machine Learning Projects
It is important to know what are A and B. Splitgraph\dbt\Dolt can work if these are structured tables in SQL database while DVC works if A and B are files or directories.
dmpetrov··on Open-Source Version Control System for Machine Learning Projects
Thanks you for warm words about DVC :)

`dvc import` can connect multiple datasets in a repository. It also preserves the "link" to the dataset that you can update/downgrade.

Class/labels distributions for a dataset can be implemented with `dvc metrics` and new feature `dvc plots`. However, you will need to introduce a dvc pipeline, like an dataset evaluation step.

Re the model zoo for data - yes, we're working on "higher level" data artifacts (like ML models on top of the basic data management layer). Stay tuned for a release soon.

PS: I'm one of the DVC creators.

dmpetrov··on Open-Source Version Control System for Machine Learning Projects
Hi @latenightcoding Dmitry Petrov here. Yes, I sent an email after we released an open source product that complements DVC. I expected that this might be interesting to DVC fans. I am very sorry if it was unpleasant for you.
dmpetrov··on Pipeline Design Patterns for Continuous Delivery
I like the post. #1-#6 patters makes total sense. I'm not sure on how #7 should be implemented. How can I create the "button" in GitLab or GitHub?
dmpetrov··on The GitHub Availability Report
External production systems depend on GitHub. For FB - it is fine to fail from time to time. Users will be even more productive :)
dmpetrov··on GitHub was down
Why this is usually happen on weekends? The only time that I have for coding :)
dmpetrov··on Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects
150Gb ML model file? That's cool!

Yes, DVC can help with that. Where the data lives? S3/GCS or just a server with SSH?

Disclaimer: I'm a creator of DVC.

dmpetrov··on Show HN: Model Zoo: Deploy your machine learning model in a single line of code
Relying on the frameworks formats seems like a great approach. However, there are a few more open questions related to model governance. Versions of frameworks, input data specification, output data/scores specification. I'd expect all of these pieces to be part of ML model description (this is even more important for model serving, then storing/versioning).

It would be great to come up with a common format for all these pieces. So, many levels of ML stack can use it.

dmpetrov··on Show HN: Model Zoo: Deploy your machine learning model in a single line of code
Neat idea.

I'm wondering what ML model formats you use and what is specification for inputs\results? Do you try to use some common format or it is fine for you to use a proprietary one?

I'm building https://DVC.org - Git for data. We think about how to save and version ML models properly (through GitFlow). One of the biggest challenges - there are not common formats for ML models, inputs and scorings formats.

I'd appreciate if you could share your opinion about models format generalization.

dmpetrov··on GitOps for Data and ML
That's funny but DVC was not started as Terraform for Data. We were just focused on data versioning and ML workflow (pipelines, metrics etc...). But it kind of becomes Terraform for Data. It is a very good analogy.

PS: I'm one fo the DVC creators.

dmpetrov··on Show HN: DVC 1.0 release. 5 lessons from 3 years of building open-source ML tool
Good question! We build separate products (that use DVC) for monetization. No plans to monetize DVC.

Some analogy - Git is free for versioning, GitHub/GitLab as monetization.

dmpetrov··on DVC 1.0 release: new features for MLOps
Hey HN, creator of DVC here!

DVC (https://dvc.org/) is known as Git for data projects. Technically, DVC codifies your data and machine learning pipelines as text metafiles (with pointers to actual data in S3/GCP/Azure/SSH) while you use Git for the actual versioning. DevOps folks call this approach GitOps or more specifically in this case - DataOps or MLOps.

We’ve been working towards 1.0 since we started 3 years ago. What began as my pet project now has 100+ code contributors, 100+ documentation contributors, and thousands of users.

Our community has taught us a lot - here are some of the biggest lessons:

1. Users say the serverless and distributed nature of DVC (inherited from the underlying Git) is one of its "killer features".

2. To share ML projects within and between teams, it’s not enough to track only files and pipelines. You also need metrics, plot and hyperparameter tracking. In DVC 1.0 we implemented hyper-parameter diffs, metrics and plot diffs right from Git history.

3. In DataOps, data transfer optimization is huge. Large deep learning models, millions of images in datasets, etc. We doubled down on optimizing 1.0.

4. ML pipelines evolve faster than data engineering pipelines and need to be easy to change. In 1.0, we’ve simplified the pipeline metafile format.

5. More and more teams use DVC as a part of CI/CD for ML and other MLOps tools. DVC is used under the hood in the CD4ML tool that was described in the canonical post on Martin Fowler’s blog: https://martinfowler.com/articles/cd4ml.html. We built 1.0 with CI/CD users in mind.

More details on https://dvc.org/blog/dvc-1-0-release.

Happy to answer any questions.

PS: I've published this in another thread, but I repeated it here.

dmpetrov··on An Open Source Stack for Managing and Deploying Models
I really believe that "end-to-end AI platforms" of the future should be built on top of open-source tools like these two. I cannot wait to see when other parts of AI platforms will be integrated with the existing ones: model performance monitoring, data catalogs and etc..

Disclaimer: I'm one of the creators of DVC.

dmpetrov··on Qri: A global dataset version control system built on the distributed web
"From a very high level perspective - Pachyderm is a data engineering tool designed with ML in mind, DVC.org is a tool to organize and version control an ML project. Probably, one way to think would be Spark/Hadoop or Airflow vs Git/Github." from https://news.ycombinator.com/item?id=19130499
dmpetrov··on DVC – Open Source Machine Learning Version Control System
Content-defined chunks - very interesting. I'd suggest you ask this question in DVC issue tracker or DVC channel https://dvc.org/chat
dmpetrov··on DVC – Open Source Machine Learning Version Control System
The major pitfall - it depends on your goal. Your approach looks good if you just need to retrain an existing model\code in production. However, this approach is not perfect in the ML modeling\development stage. Let me explain...

`--entrypoint` defines a single step\script. You make a single black box from your entire ML processes. There is a lack of granularity for ML modeling process when people tend to separate different stages to make the process more manageable and efficient: manage and version dataset separately from modeling, preprocess data before training, training code as a separate unit plus some problem specific steps\units.

DVC gives you the ability to increase the granularity of your ML project while still keeps it manageable and reproducible. The steps still can be wrapped in Docker - it is a good practice. As @theossuary said, run `docker run -ti experiments:latest` as a step in DVC.

dmpetrov··on DVC – Open Source Machine Learning Version Control System
Exactly. And it is flexible as Git. You can define your own workflow. For example, some data scientists avoid using git-branches for experiments - they use directories.
dmpetrov··on DVC – Open Source Machine Learning Version Control System
The sweet spot for LFS is a few 100Mb files with less than a couple Gb overall. While DVC was designed with 10Gb-100Gb scenarios in mind.
dmpetrov··on DVC – Open Source Machine Learning Version Control System
Great point! `dvc pull\push mydataset` actually exists in DVC. But `dvc tag` is missing and now we (DVC team) understand that it prevent DVC from a proper datasets and ML models tracking. It will be implemented soon.

There is an ongoing discussion in DVC GitHub about datasets tracking and tags https://github.com/iterative/dvc/issues/1487 and some discussions in DVC discord channel https://dvc.org/chat

dmpetrov··on DVC – Open Source Machine Learning Version Control System
Docker can help only if there is a single step in your project. In ML projects you usually have many steps - Preprocess, Train. Each of the steps can be divided: extract Evaluate step from Train etc.

Also, Docker has an overhead - copy of data needs to be created. While DVC just saves links (sym-links, hard-links or reflinks) with a minimum overhead. It is crucial when you work with multi-GB datasets.

dmpetrov··on DVC – Open Source Machine Learning Version Control System
Example. A query to DB gives you different results since the data\table evolves over time. So, you just store the query output (let say a couple GBs) in DVC to make your research reproducible.

This is like assigning a random-seed to DB :)

Sure, some teams combine DVC with AirFlow. It gives a clear separation between engineering (reliability) and data science (lightweight and quick iteration). A recent discussion about this: https://twitter.com/FullStackML/status/1091840829683990528

dmpetrov··on DVC – Open Source Machine Learning Version Control System
@mcncfie your understanding is correct. #3 might include output data\models as well and intermediate results like preprocessed data. DVC also handles dependencies between all of these.

Answers:

1. DVC does dependency tracking in addition to that. It is like a lightweight ML pipelines tool or ML specific Makefile. Also, DVC works just faster that LFS which is critical in 10Gb+ cases.

2. This is a great case. However, in some scenarios, you would prefer to store the query output along with the query and DVC helps with that.

3. Correct, there are no data diffs. DVC just stores blobs and you can GC the old ones - https://dvc.org/doc/commands-reference/gc

← PreviousPage 2 of 3Next →