I would napkin math the averaged cost per-review to be less than $1, which is less than our most rigorous validation pathway in CI (needs GPU for up to an hour). Time-wise around ~10-15m, with variability - and a bug in Argo with pod scheduling delays. This is fine because developers check in on their own PRs when they do and then make the changes that are needed. A key point here is that we are keeping humans in the loop in the PR.
This is a large monorepo, I would expect much lower numbers for polyrepo setups.
More or less, I did:
- Avoid unnecessary work
- Don't boot more often than needed
- Parallelism in tests
- Caching rules
My post has an ongoing live survey with ~570 responses:
- 30% says 5-15m
- 26% says >30m
- 23% says 15-30m
- 13% says 2-5m
- 8% says <2m
was around 40 mins on the critical path at the start of the year. gains came mostly from sharding tests, replacing webpack with vite, eslint/prettier with oxlint/oxfmt, caching node modules