So I think I understand what is meant, but I don't like the comparison.
System one was an attempt to distinguish the "thinking" of thinking models (which is more akin to system two thinking with deliberate arguing/building up a decision) from fast instinctive thinking. It's a really good metaphor for the kind of trade-off or use case this applies to.
It is not and was never implying that it is a replacement for all human system one thinking activities.
Maybe the people talking about an AI bubble do have a point. I am getting the impression here that investors are throwing money at everything.
Personally I believe AI labs have a solid business model that could soon be very profitable, but this here has me doubting now.