Because at the limit, the size of the pool is one...you.
When you buy insurance, you aren't just pooling risk with other customers with the exact same risk profile as yourself. You're pooling with all customers of the company. (Plus all customers of any reinsurance company, etc.)
Put another way, if the insurance company isn't taking on any risk then you are.
Obviously with something like medical insurance, where people often carry risks that are no fault of their own, there are downsides.