Moving fast is often crucial, but you typically also get more readable code, which is also worth it.

A query/DML that is badly optimized is usually very complex, which means that the ORM's syntax isn't a good fit. No problem, since they typically support raw SQL calls, so nothing is really lost.

You just have to know when to use the right tool for the job, as always.