Yes. Infinitely large sets are decidedly unphysical, and taking the metaphor too literally can be misleading.

Imagine: how could the proprietor even communicate with those infinite guests?