I'm well aware of those two popularly perceived problems, but I don't see how either justifies him being a bad PM.

I believe proposition (1), there being more evidence that supports this view and more rhetoric that supports (2).

If he were elected with a significant majority of both houses, and the party returned to a more centrist state, such that he didn't need to appease the more Right-wing faction of his party in the same way, would he not still be the perfect PM many of us thought he might be?

An aside -> his election strategy was mostly: (1) use data to find popular buzz-words ("JOBS", "GROWTH", "INNOVATION"); (2) repeat those ad infinitum; (3) shy away from any compelling political narrative or vision for the future of the country (e.g. "MAGA" narrative); (4) ???; (5) Lose House seats to Shorten and Senate seats to Pauline et al.