This is one of those that unnerved me as well the one time I drove a Tesla - I don't know if the display is accurately representing the car's model of the world around it, but it seems to me that you'd want to assume that real-world objects don't just disappear when you can't see them anymore; they have a size and shape and velocity and you can estimate their location based on relative movement (with, of course, ever-increasing inaccuracy) even if you can't clearly see or identify them at the moment.
Though, it seems like such a basic thing that there's probably some tradeoff that I'm not thinking of.