No, those $100 tickets are mostly worth $100. There are a lot of reasons why someone will overpay to get a ticket; but don't delude yourself: If Taylor Swift decided to charge $1000 for each ticket, many less people would go. She might sell 1/10th of the tickets, and it would be a wash, she might sell 1/3rd of the tickets, and make more money, or she might sell 1/20th of the tickets, and make less money.
What drives the price of the tickets up is scarcity: Once the venue is sold out (or close to sold out,) it's useful to only sell the tickets to people who really, really want to pay.
Edit: I should add that, unless I really want to see an artist, I tend to buy my tickets shortly before the show, and only if they are a reasonable price.
For example, last summer Green Day & Smashing Pumpkins was $200 / ticket. I kinda wanted to go, but I didn't want to pay that much.
In contrast, Weezer was $70 a ticket and playing around the corner from my office. I went to Weezer. (And wished I bought a floor ticket sooner because the Flaming Lips were the opening band.)
Had Weezer charged $200, I never would have gone.