1,434 karma · joined July 17, 2010
[1] https://iterative.ai/blog/experiment-refs
[2] https://marketplace.visualstudio.com/items?itemName=Iterativ...
I 100% agree that managing large datasets by moving them around is not practical, and definitely not in LFS/DVC-style. There should be a level of indirection if reproducibility is needed (pointers are versioned to files, not the data directly, data should be staying in the cloud).
Here, I would love to one more time mention some other cool features that DVC has. E.g. `dvc exp` set of commands where it is creating custom git refs to snapshot experiments, of DVCLive logger that helps capturing metrics, plots, etc. And also VS Code extension [1] that provides quite cool experience for experiments workflow inside VS Code.
Point here is that for DVC the ability to capture some large files and directories (that do not fit into Git) was always a low level mechanism to support higher level scenarios (e.g. you need to save a model somewhere as an output of an experiment).
[1] https://marketplace.visualstudio.com/items?itemName=Iterativ...
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (12K+ stars), SaaS product - studio.iterative.ai.
We are looking for senior (SaaS and open source) engineers:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (11K+ stars), SaaS product - studio.iterative.ai.
We are looking for senior (SaaS and open source) engineers, also for product engineers:
In most case in these companies it's maintained and created at the expense of the company. I would expect that 90-99%% of the code, product development (talk to users, understand needs, etc), even devrel (marketing) (be on every conference to convince people to use it, that codification is good)- it's a lot of money, resources, and effort.
> New startup, cool idea, not much budget to hire engineers
In this case a few folks build it from the ground initially (most likely founders) I think. Please, let's not forget about this.
The important thing here is to discuss why did they do this (I meant relicensing it). Most likely- trying t create a moat from a lot of other VC-funded companies that play in this space? Not sure. It would be great to know their exact concern.
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers (MLOps). We need people who are passionate about building infrastructure that standardizes how ML teams manage data and models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (11K+ stars), SaaS product - studio.iterative.ai.
We are looking for tech leads (SaaS and open source DVC.org) engineer:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers (MLOps). We need people who are passionate about building infrastructure that standardizes how ML teams manage data and models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (11K+ stars), SaaS product - studio.iterative.ai.
We are looking for tech leads (SaaS and open source DVC.org) engineer:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers (MLOps). We need people who are passionate about building infrastructure that standardizes how ML teams manage data and models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (11K+ stars), SaaS product - studio.iterative.ai.
We are looking for a senior Python (backend or systems programming) engineer:
Also senior ML Solutions engineer (like internal senior ML / MLOps consultant or a customer facing engineer who can code and enjoys helping other people every day with their needs). Remote is okay. Excellent communication skills, english:
Apply directly ivan AT iterative.ai. For a successful, personal (no agencies) recommendation we do 10K bonus.
More about us here:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers (MLOps). We need people who are passionate about building infrastructure that standardizes how ML teams manage data and models. We are building infrastructure to deal with datasets of the 5B laion size. Our flagship open source tool- DVC.org (11K+ stars), SaaS product - studio.iterative.ai. We are looking for senior Python (backend or systems programming) and product designer. Please, find job descriptions and apply here:
https://jobs.lever.co/iterative/a1f9061e-112c-4b6a-847b-38b2...
More about us here:
I love the experience so far, we've done a few features (modules) - e.g. this one to install `nvtop` to see GPU utilization [0]
The whole CUDA + nvtop + (some other tools) setup for a project to be run on a remote machine (e.g. via VS Code) becomes like this [1]
And that's enough to run ML training on GH Codespaces with GPU support. Super cool experience.
[0] https://github.com/iterative/features/blob/main/src/nvtop/in...
[1] https://github.com/shcheklein/hackathon/blob/main/.devcontai...
Happy to provide more details on how it's done. It's actually quite interesting technical thing - custom Git namespace https://iterative.ai/blog/experiment-refs
It's been working really great. GitHub PRs + preview applications give a very powerful collaboration / review framework. It works extremely well for folks who understand Git / GitHub enough - engineers, technical writers, etc.
It's starts breaking unfortunately for the marketing team, less tech savvy folks. I was trying to find a solution CMS that would work for them, but haven't seen so far something that would satisfy our needs. Two things that we are usually missing:
- we need a way for people to come and review it (ideally GitHub / Notion / GoogleDoc style). Where you would be able to comment on a line, suggest and edit, etc, etc - we need a way to handle assets better (using Git for blog was not scalable - started breaking the workflow). So far we ended up using DVC for this, but it complicates the workflow.
- there are merge conflicts that are happening - that's not easy to resolve in CMS
Nonetheless, even a tool that would prepare and generate a PR using an online editor is already a very decent step to my mind. Eager to try the tool.
Not sure if in this case it's something different but my take from the past experience it's hard since data is noisy + closing the deals depends on a human as well and these tools don't take that into account usually.
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Open source dev tools for ML engineers and Data Scientists (MLOps). We build DVC.org (10K+ stars on GH), CML.dev (3K+ stars on GH), SaaS product - studio.iterative.ai. Think about us as a Hashicorp for ML and MLOps. We are looking for senior Python (backend or systems programming) and front-end senior engineers. Please, find job descriptions and apply here:
https://jobs.lever.co/iterative/30cb96fe-e210-4a46-b0eb-ea96...
More about us here:
Another difference is that for DVC (surprisingly) data versioning itself is just one of the main fundamental layers that is needed to provide holistic ML experiments tracking and versioning. So, DVC has a layer to describe an ML project, run it, capture and version inputs/outputs. In that sense DVC becomes a more opinionated / high level tool if that makes sense.
Would love your feedback what's missing there! We've been improving it lately - e.g.
- Hydra support https://dvc.org/doc/user-guide/experiment-management/hydra
- VS Code extension - https://marketplace.visualstudio.com/items?itemName=Iterativ...
Also, one thing is to physically provide a way to version data (e.g. partitioned parquet files, cloud versioning, etc, etc), but another one is to also have a mechanism of saving / codifying dataset version into the project. E.g. to answer the question which version of data this model was built with you would need to save some identifier / hash / list of files that were used. DVC takes care of that part as well.
(it has mechanics to cache data that you download, make-file like pipelines, etc)
https://docs.brew.sh/Analytics https://docs.npmjs.com/policies/privacy#how-does-npm-collect... VS Code, etc
> I think the challenge I have is that since you’re getting IP address that will be an opportunity to abuse.
Yes! And we are migrating to the new package / infrastructure because of this - https://github.com/iterative/telemetry-python (DVC's sister tool MLEM is already on it and it's not sending (saving) IP addresses, nor using GA or any other third-party tools, data is saved into BigQuery and eventually we'll make publicly accessible - https://mlem.ai/doc/user-guide/analytics to be fully GDPR compatible). It's a legacy system that DVC had in place. There was no intention to use those IP addresses in some way.
> I think perhaps the only other way would be to support an automated distro that doesn’t include it so users are at least able to easily choose a version.
Thanks. To some extent brew-like policy (not sending anything significant before there is a chance to disable it and there is clear explicit message) should be mitigating this, but I'll check if it works this way now and if it can be improved.
We've tried to make it as open as possible - code is available (its open source), we write openly about this at the very start, we have a policy online, made it easy to opt-out. If you have other ideas how to make it even more friendly, more visible, etc - let us know please.
Still, we've preferred so far to keep it opt-out since it's crucial for us to see major product trends (which features are being used more, product growth MoM etc). Opt-in at this stage realistically won't give us this information.
It should be easy to opt-out though `dvc config core.analytics false` or an env variable `DVC_ANALYTICS=False`.
Could you please clarify about the `several lines of code`? We were trying to make it very open and visible what we collect (it prints a large message when it starts) + make it easy to disable it.
Fun fact: SimCity was forced to pretend that all parking lots were underground, because the game would be “really boring if it was proportional in terms of parking lots". SimCity’s lead designer explained, “I was blown away by how much more space was parking lot rather than actual store. That was kind of a problem, because we were originally just going to model real cities … We had to do the best we could do and still make the game look attractive."
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Open source dev tools for ML engineers and Data Scientists (MLOps). We build DVC.org (10K+ stars on GH), CML.dev (3K+ stars on GH), SaaS product - studio.iterative.ai. Think about us as a Hashicorp for ML and MLOps. We are looking for senior Python (backend or systems programming) and front-end senior engineers. Please, find job descriptions and apply here:
More about us here:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Open source dev tools for ML engineers and Data Scientists (MLOps). We build DVC.org (10K+ stars on GH), CML.dev (3K+ stars on GH), SaaS product - studio.iterative.ai. Think about us as a Hashicorp for ML and MLOps. We are looking for senior Python (backend or systems programming) and front-end senior engineers. Please, find job descriptions and apply here:
https://jobs.lever.co/iterative/30cb96fe-e210-4a46-b0eb-ea96...
More about us here:
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Open source dev tools for ML engineers and Data Scientists (MLOps). We build DVC.org (9.5K+ stars on GH), CML.dev (3K+ stars on GH), SaaS product - studio.iterative.ai. Think about us as a Hashicorp for ML and MLOps. We are looking for senior Python (backend or systems programming) and front-end senior engineers. Please, find job descriptions and apply here:
More about us here: