For example, if we are measuring how light bends - how do we know there aren't many hidden (i.e. not large or bright enough to be noticed from earth) objects causing additional distortion and therefore throwing off our measurements? Do we also observe it over time to get more confidence, or have some other cross-checks that give us more certainty?
Either way, every once in a while I find it wild just how much we can deduce about the universe from our tiny fixed vantage point.