The big costs are staff and rent. I was referring to the markup from the factory cost to the sticker price - that is very high in clothing. It can still be money losing business if you are not selling enough.
It is only because the retailer cares about factory cost that the retailer can afford the staff and rent. Even paying the relatively low factory cost, the retailer can barely eke out a profit, so they must be discerning about the price to quality ratio of the clothing they are buying to retail to the public.