That's not the only factor at play. Many to most applications are dominated by single thread performance. JavaScript's interpreters are single threaded, for example.

Something I don't often see, but it does come up here and there. One nice thing about the M1 is the performance is consistent as it doesn't have a massive auto-scaling boost involved. An Intel or AMD chip might start off at top speed single thread, but then something else spins up on another core, and you take a MHz hit on your primary thread to keep your TDP in spec. The background task goes away, and the MHz goes back up. Lots of performance jitter in practical use.

Interconnects and IO also consume power. You can't just scale small e-core counts without also hitting power walls there too.

All that said, I'd love to see some E-core only chips come out of intel targeted at thin clients and long battery life notebooks.