Which is insanely far away and all light coming from there will have diminished in energy enough to be barely visible?
Inadequate explanation. Even if the universe was a closed box with perfectly reflecting borders, what you would actually see in the sky is something like an average brightness of the entire universe with some fluctuations.