Yup, this is the main reason I won't be upgrading to Windows 11 anytime soon. I've never been a fan of the icons-only taskbar and have enabled text labels since Windows 7.
I won't be upgrading for this reason alone. This is not about being used to the old way and not wanting change, it's about not wanting to switch to something that is objectively worse and will make me less productive.