This way of thinking is why every commercial software package looks the same. Why must every background and panel be grey? Why must we have only one primary colour? Why must every colour field have edges?
Ignoring these guidelines can easily and quickly devolve into something that is tacky or even garish.
Probably because these guidelines have been "proven to work" to produce a usable UI. The priority of commercial software isn't usually to produce a work of art, it's to produce a usable tool, there's little incentive to take creative risks.