No, you're not alone. There's actually an entire page maintained by a few economists on the case for open borders: http://openborders.info