It's not that the president is the sole power in government! it's that he's the sole power in the executive branch
This means the checks and balances are Congress vs Court vs President. Just like the clerks for SCOTUS don't have any power and neither do the staffers or clerks of Congress, so too do all the members of the executive branch have no power, except for whenever the Pres decides to delegate his power to them.
Even those who argue with this idea (few do) the only other ppl who have power in executive branch are the officials who get confirmed by the senate bec. they are mentioned in the constitution. No one thinks that unelected bureaucrat have any standing whatsoever outside of what's been delegated to them by the President
This does not in any way pertain to ruling by EO! Many things should in fact be done by Congress, but those that can/should be done by the executive branch are under the Presidents full control.