If you never helped no one and was in the most perfect utopia all the time and now want to go to a below average world (probably for nostalgic reason, it was the world of your childhood), others probably won't be so eager to join "the party". So you'll probably have to help someone else live in the below perfect world of their childhood first.
You see all world in a static fashion no problem or explore them with your powers but if you really want to forget it all and become a simple human in some a bit dystopian world, you'll need others to join you ("vote").
So people with multiversal powers (to recall and forget are the main ones) have 4D spacetime of each verse.
Easier to imagine 1 universe or 1 planet: the whole spacetime as a giant walkable long exposure photo but in 3d (when I say "4d" here, I mean it has many moments of time aligned on top of each other to make it look like a long exposure photo where you can focus on a moment or defocus and see billions of years all at once).
Multiversal people can basically change the shape of the multiverse and change the shape of themselves by forgetting parts of this giant geometric shape. If two or more people forgot the same parts, they "direct democratically voted" to be together there for some time, chose to visit the "same party".
Basically you're the 4D simulated multiverse and you can temporary choose to forget this fact to become some 3D slice of it that experiences the illusion of time (others can do it, too).
Almost infinitely many geometric shapes have almost infinite freedoms to change their geometric shapes instantly or slowly if they wish. To temporarily slice/forget parts of the multiverse. Anyone can choose to permanently die, too, but cannot kill their baby version, they are like eternal time loops.