Is this a rhetorical question? Yes, it is the world I want to live in. If the business operates within the context of healthy competition, then while they are free to refuse service, others are equally free to boycott it. This democratizes social norms. I don't want to force gun shops to sell to people they have a bad feeling about (say, because they have a racist tattoo), and I don't want a law that attempts to distinguish between "good" iffy feelings and "bad" ones.
If on the other hand the company is a monopoly and no reasonable alternatives are available, then it is de-facto infrastructure and should be more stringently regulated for impartiality, as is the government (and frankly such companies are a problem anyway, and should possibly be adopted by the government when they reach that size). The DMV isn't allowed to turn you away for a swastika tattoo, and neither should the electric company. But a baker? Absolutely.