Before our modern understanding of cosmology, a serious question astromers asked is "why is the night sky dark". If there truly were stars/galaxies in every direction you would expect the entire night sky to be bright, as no matter which way you looked you would see a star; and the combined light of all of those stars should illuminate the sky.
As it turns out, empty space isn't dark, but rather illuminated by the cosmic microwave background. This suggests that the space between us and this boundary in space from when photons started to be able to move mostly freely is mostly empty. Otherwise this backround would either be over-shadowed by stars, or obscured by whatever stuff was obscuring said stars.
Of course, it is important to keep in mind the speed of light implications.