In that case, if you pay a guaranteed -$1, your expected utility loss is U(-$1). If you have a 1e-6 chance of losing $1e6, your expected utility loss is 1e-6 U(-$1e6) < U($1).
Thus, it makes sense to pay $1 to avoid the risk of losing $1e6.
Insurance which pays for high probability, low cost events (e.g., gas for your car, birth control pills) is indeed foolish.