Which people are these that "think of the Matrix" when they think of VR? I've seen some people talk about it, or more often Sword Art Online. But they are rare. It tends to mark folks as not having really tried VR. I've seen people talk about not even wanting to try VR until it's "full dive". That's dumb. I doubt it's ever going to get there and I also doubt it's even a good idea. See: The Matrix and SAO for depictions of why.
I think "people" know the tech is not magic and have decent expectations on what it entails. What I tend to see are people blown away by how impactful that simple "box with a display and some lenses" concept works at transporting a person to a new world.
I don't think anyone really cares about black levels that much, not to the degree that it becomes any kind of deciding factor on purchases. Focus... are you sure you're not Ronny Abovitz? Literally the only person I've seen ranting about focus. Again, nobody cares. It'd certainly be nice, but it's just not that important of a feature.
Your comment on latency is confusing. Which latency? Motion-to-photon latency? Hasn't been a problem in years. Network latency? We're not talking about a networked game here, and regardless, that's going to be true for any kind of multiplayer experience, VR or otherwise. It's also a largely solved problem and people don't tend to notice network latency in most multiplayer games. The only latency issue I encounter on a regular basis is Oculus Quest 2's hand tracking latency. That's not a fundamental limitation, though, because other hand tracking systems like UltraLeap show that it can be done very well. The Quest's problem is that it's running on what's essentially a 3 year old smartphone SOC.
As for "killer apps", honestly, I have never really believed in the concept. There's nothing on my smartphone that I can't do without it being specifically on my smartphone. But combined, the whole ecosystem is extremely useful. My smartphone also isn't the seamless digital assistant Jobs predicted it would be, but it's still extremely useful. Back in 2008, most people thought smartphones weren't for them, but now most find them extremely useful.
There are problems in the VR industry. Any device that isn't the Quest 2 is too expensive and not featureful enough to warrant the expense. Building software for VR is still extremely primitive (No, Unity doesn't solve it. Come back to me after you ship a Unity VR project that meets Meta's store guidelines if you disagree). Battery life on standalone systems is abysmal. I don't know anyone with a PC-tethered system that hasn't destroyed several cables, ports, keyboards, or even just glass containers.
And there are problems being foisted on VR. The global chip shortage is making it way to hard to scale. Meta's attempts to dominate the market by undercutting the competition is a huge danger to everyone. This coopting of social VR into "metaverse" by crypto grifters is going to kill an entire segment of the app market. Megacorp's inability/unwillingness to moderate user behavior on their platforms is a lot more harmful to people's mental health when it's literally in their faces.
But the biggest one might be people who don't know anything about VR claiming it's "useless" or "dead".