The only caveat is that it's important that you work in an organization that is aligned with remote work, if not remote-first. Just scoring 'permission' to work remotely somewhere not used to it, is probably a recipe for a pretty bad time.
I really agree with your caveat, and I've seen a number of places that have some pretty unmotivated individuals try it, and have it fail. This left a bad taste in everyones mouth, and they wrote off remote work. The reality was that they had crappy employees, and not that remote work can't work.
Why is that?
Hence the idea that remote working is probably most effective when companies treat it as the norm--at least on a group by group basis.
I've actually never worked at a place with a large number of devs under the same roof.