There's plenty of scope for further discussion about these points, but the key point remains: shareholder primacy is not law and not the focus of company directors' fiduciary duty, regardless of what people claim in self-serving arguments from various points of the ideological spectrum.
Uncapped growth is just as evil as unfettered greed - both charge forth without a second thought to externalities, while motivated to externalize as many costs as possible. So we end up with boiling oceans and rampant homelessness, because those are all "not our problem".
They are not. They are expected to serve the interests of the company, as distinct from the interests of shareholders, and matters like employee welfare/satisfaction and company reputation relating to externalities are very relevant to the interests of the company.
That some companies ignore these matters is separate from the principle: it’s not inconsistent with capitalism for companies to care about issues other than profits and shareholder returns. Though it’s obviously related, company welfare is distinct from shareholder benefit.