The societal impact of that UI design decision will be interesting to watch play out. People are so used to trusting the first Google search result that it now being AI that's sometimes wrong or hallucinated.
Frankly, the AI section at the top seems like something Google would have been very reluctant to add since it saves the user from scrolling through the ad listings, and it was only added to compete with newer AI search services.