All our current physics says that the speed of light is a real limit to travel, and to get close to that requires far more energy that a society is likely to have. I can well believe that aliens if they exist may conclude leaving their solar system is not worth the cost. Sure they could do it, but many will die when their spaceship on the way. Even robot spaceships are unlikely to make it very far (on the scale of the galaxy) before suffering a critical failure.

Thus I can well believe that aliens have colonized the other habitable planets in their solar system and failed at all attempts to do anything more. They might have got one robot to a different star, but that is it.

Of course this assumes they are there before us by a significant amount. Maybe humans are first to reach our level - it is an arrogant idea but we cannot completely discard it until proven wrong.