Have you tried AR kit on a new iPhone? The tracking is pretty damn good and that’s just with 2 cameras.
Two calibrated cameras and, more importantly, calibrated accelerometers and gyroscopes. Cameras alone aren't fast enough, and visual feature detaction not accurate enough, to provide a good experience.
All of this is within reach of anyone who is making their own hardware, though.
Google/Lenovo managed it with the Mirage Solo on "fairly lose to stock" Android.
Actually just one. ARKit works on nearly every iOS device since the iPhone 6S.
Hololens has been doing inside out tracking on much older hardware for years now. It's not impossible.