So aesthetics comes into building their own distinctive "brand" (as opposed to the brand of the client/company they are currently working for). That's not to say they won't justify their design in terms of what the client needs, it just pushes them away from using "boring" aesthetics - for example, pushing for a trendy-looking design for a backend admin dashboard that would have been just as usable (and cheaper) to build with a bog-standard Bootstrap theme.
This isn't to blame designers, that's just the economic reality of the business.
Modern UI/UX zeitgeist is driven in large part by the web and mobile, which are driven by the "attention economy". The defining aspect of attention economy is that it makes money on friction. The longer it takes you to accomplish your goals, the more opportunities there is to display you ads, or upsell you something, or tire you out and make you more susceptible to be funneled to where the vendor wants you to be. The major design guidelines and UI frameworks are written and provided by advertising companies, and are focused on controlling the interaction, not empowering users. In this environment, ergonomics and good experience - meaning a tool that helps users achieve what they want as fast as possible, and gets out of the way otherwise - is actively discouraged.