Yes, but with something CSP-derived, you could design a runtime where you're covered if you use the concurrency facilities correctly, and you're SOL if you don't. Then only the concurrency primitives need atomic refcounts.
I'm managing something like this in Go. There are no refcounts, but everything is very much mutable. I'm basically arranging for a span of time where I know nothing unprotected by a channel is going to be mutated, then I simply let every thread in the app that cares to read data from every part of the heap, but only during this span of time. The same technique could be applied to a ref counted app. (It would probably work best for games that have a tick.)
I still think it would be hard to apply to a ref counter app, since you'd need to keep track of change in ref count for later cleanup (thread-local per object maybe? sounds inefficient), but I now will admit that it sounds possible.