The basic problem with your suggestion is that it does not explain how anything actually "happens". Unitary evolution is a theory where nothing changes. People seem to think that they can evoke decoherence, or similar, and these issues disappear. But they do not.
Anyway, I'm not positing any solution, I am just saying that it irks me that all these ideas are dismissed whenever it comes to the problem of the asymmetry of time. It just seems bizarre to me.
<u|U^\dag U|v> = <u|v>.
OTOH, I can understand the argument that nothing ever happens. One reasonable requirement for something to happen is that it should not be able to "unhappen," which is why some are looking for the possibility of irreversible decoherence.
This is not unitary. Quantum theory has two parts: unitary evolution, and measurement processes. That's it.
Let u and v be qubits. Apply a Hadamard gate to u while leaving v unchanged. Then they have evolved independently (but unitarily): u according to H, and v according to I.