And while I believe other types of slowness make users unhappy without seeing the literature, the people driving our priorities don't see the $ impact at all
But speed is a visitor's feature, not a benefit. The benefit (to the web site operator) of an ecommerce site is $ spent by visitor.
So for example yes, the Amazon site feels slow and bloated, but they've been doing live A/B testing for over 20 years and I'm pretty sure they don't care about speed unless it affects conversion; they'll be happy to cram something in even if it slows the site down, if it increases revenue.