You are not alone. The last time a home internet upgrade felt really significant to me was going from 2mbit to 8mbit. Everything else has been luxury. Though I'll be signing up for ~160 or ~1000 as soon as the new cables the street was recently dug up to put in place are connected, I like my luxuries!
I do still notice upstream improvements though. Both those old 2mbit and 8mbit connections were 256kbit up, and I currently have "up to 17mbit" which is [quick check of router logs] ~11mbit ATM - that can be limiting for backups or when wanting to share video or HQ photos with a large group. So the FTTP upgrade _might_ be justifiable as more than pure luxury.