Most of the people who encountered the N64 did not know how to hold the controller at first glance. This was made worse by the fact that practically every game has a different idea about how the controller was to be used. Nintendo had to settle a court-case by distributing gloves to players complaining of friction injuries.[0] .
This was mostly because developers didn't know how to deal with the control schemes of 3D environments very well, and Nintendo was planning for what they considered to be needed in the future : more controls.
There's very likely a reason that the M shape was never used for controllers again, however it did leave a lasting legacy in the form of trigger buttons (not shoulder buttons), which are now practically a universal controller feature.