We tend to recommend folks spend a few hours writing evals after they spend a couple weeks prototyping. Then they get a sense of how valuable evals are for their use-case.

We think about evals a bit like perf monitoring -- it's good to have RUM but also good to have some synthetic stuff in your CI. So if you do find them valuable, useful to do both.