We migrated all our devs to colima and it's been pretty good. Don't want to deal with Docker (the company) and don't use any of the "features" of Docker Desktop.
The only actual issue we've seen is the way folder syncing works includes a slight delay, which has caused some unit tests that depend on running commands in a container with the current working directory mapped that create files to fail intermittently. We believe this is because of sshfs caching. We've updated the tests to wait for the files to appear rather than assume.