It falls short because you can just transpose the latents and use whatever vae you want.