I followed advice of Sindre Sorhus [1] and moved all my packages and apps to ESM year ago and couldn't be happier. Only Jest and ts-jest were problematic, so I replaced them with Vitest. I also never encountered problems because of the so-called dual-package hazard [2]; but IMO this isn't that much different than when you have two copies of React in node_modules - it's simply an npm/dependencies problem, not ESM problem.
[1] https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3...
[2] https://nodejs.org/api/packages.html#packages_dual_package_h...