Meta's glasses are "normal sized" because they don't even attempt to do what headsets do. To think one will replace the other misunderstands both.
You could add more cameras and lidar for depth, yes, but then you'd also need a display for the other eye, and you still wouldn't be able to do immersion because your displays are transparent and have a tiny FOV, and you're also not doing passthrough, so why do you need the cameras again? And so on.