Either quantum computers pan out over the time investors stay interested or they don’t. It’s not unlike any other technology. As far as physicists feeling embarrassment, why should we? One, we’re largely incapable of feeling it, and two, we tried and that’s cool.
You have these VCs who have a very narrow view of how the future should be (chiefly being that which makes them richer) so you can only pitch so many technologies that fit that mold. To be blunt, find me a better technology that could potentially push the boundaries of human capability or understanding or whatever that VCs will invest in. Beats the hell out of VR. Is it cooler than shooting stuff into orbit? Seems on par to me.