Many people willing to pay that much for a polo shirt consider lack of conspicuous brand to be a feature.
Leaving a logo off the polos basically makes any sort of viral growth impossible. Of course the plan could be to sell shirts exclusively to wealthy people that like the "lands end" look. That doesn't strike me as the best approach.