Why can’t we look for moving dim spots in the cosmic microwave background?
Try looking out the back of your head for the planet hiding right behind you. Once you figure out that you have to turn around, the planet might not be there anymore :) We could build telescopes & point them in every direction though!
Well, yes. True. Not spectacularly informative though.