By an order of magnitude or two, yeah. In addition to the very pricey speakers and receiver, you also, at that point, need fine control over room shape and acoustics, as well as more space than any apartment is likely to provide, in order to get the benefit of what you're paying for. Headphones are a much more controlled and thus easily engineered environment, so it's a lot cheaper and easier to get equivalent quality, albeit with a necessarily narrower soundstage in almost all cases.
The $450 I paid for my endgame headphone/amp/DAC wouldn't even get you in the door of an audiophile speaker place.