> You are encouraged if you model your pipeline in DVC.
Encouraged to do what?
You might want to slow down on the use of parentheses, we are both getting lost in them.
Encouraged to do what?
You might want to slow down on the use of parentheses, we are both getting lost in them.
However it's fantastic for tracking artifacts throughout an project that have been generated by other means, and for keeping those artifacts tightly in sync with Git, and for making it easy to share those artifacts without forcing people to re-run expensive pipelines.
Would love your feedback what's missing there! We've been improving it lately - e.g.
- Hydra support https://dvc.org/doc/user-guide/experiment-management/hydra
- VS Code extension - https://marketplace.visualstudio.com/items?itemName=Iterativ...
Ideally I'd like the tool I use for data versioning (DVC/git-lfs/gif-annex) to be orthogonal to that which I use for hyperparameter sweeping (DVD/optuna/SageMaker experiments), and orthogonal to that which I use for configuration management (DVC/Hydra/Plain YAML), to that what I use for experimental DAG management (DVC/Makefile)
Optuna is becoming very popular in the data-science/deep learning ecosystem at the moment. It would be great to see more composable tools, rather than having to opt all-in into a given ecosystem.
Love the work that DVC is doing though to tackle these difficult problems though!