I think of the rvalue reference as talking about the object itself rather than how you get to it (i.e. the actual bytes on a chip someplace as opposed to its address or a copy of an object). Hence moving, perfect forwarding, etc are roughly-synonymous ways to think of managing that object, maybe just looking at the same semantics from different angles.
This is just a conceptualisation, of course (rvalue definition is a little more subtle), but for me as a Lisp programmer it seems natural and intuitive. If you come from a different background or perspective I understand that it can seem weird — again, most languages are not as expressive.
The low level nature that is fundamental to C++, even when you are programming at a very high level, which you now conveniently can, is a consequence of it working hard to be a zero-cost-abstraction systems programming language, leads to issues like this one, or to some odd consequences of move constructors.
There are alternative points in the solution space. Compare it to go, which explicitly papers over some of these trade offs, making an opinionated choice instead, and being willing to embrace reduced expressive power (so there are some things you just can’t write in it). I don’t like go myself, but that doesn’t mean people who choose it are fools.