Yep. But, in most senses, trading algorithms don't need to interact with humans, or human society. Is the world around the "thought experiment" AI CEO dehumanized enough that that's also the case for it?

If not, I'd expect major problems there.

A related point - trading algo's can completely pivot in seconds, and live & die by extremely simple and short-term metrics. Even in a human-free world, that's not at all how the vast majority of industries work. What is "profit", when your AI CEO is organizing the construction of a new steel mill? Judging its performance then requires deep domain expertise.