In practice, I type those characters on their own much more often than I type them for French characters, so having to press more keys for the French ones makes more sense than the other way around.
There's also the fact that I have to type those characters in places where I don't have fancy entry systems (drive unlock passwords, rescue environments), while I don't type French in those places, so it also makes more sense for muscle memory.