I think you are right that the skills to be a successful CEO are rare, but I think the main problem is that the hiring process for CEO is so incestual that the majority of the people who would be good for the job aren't even considered. Instead the applicant pool consists of "son of current CEO", "son of CEO's favorite politician", and several "CEO that is currently failing a different company". The applicant pool feels small because they never look beyond the boundaries of "billionaire families who go to the same country club".

There are a few CEOs that seem to truly make a difference, and maybe those would be worth the money, but for the most part companies are grossly overpaying for people who are mediocre to terrible at the job and will deploy the golden parachute in a couple of years to go fail at a different company.