If the shirt lasts 4x as long, on average, its worth nearly 4 shirts. A 25 dollar shirt is not crazy, why should 100? Its all short vs long term thinking, coupled with lack of product knowledge.
Nevermind that cheap T-shirts can last a long time already.
There's just no value in spending more to make a shirt 'better'. The only reason to do it is as a status symbol.