Yes rates were 16% in 1980 but the median home was $64,000. That's $230,000 in today's dollars.
If you'd prefer to pick a time when rates were more comparable, how about 2001 at 7%. The median home price was $180,000. $301,000 in today's dollars.
Today's median home is selling for $440,000. It's small wonder that people are upset.
[1]: https://fred.stlouisfed.org/series/MORTGAGE30US [2]: https://fred.stlouisfed.org/series/MSPUS