From [0]:
"The controller is the same 18-channel behemoth running at 400MHz that is found inside the SSD DC P3700. Nearly all client-grade controllers today are 8-channel designs, so with over twice the number of channels Intel has a clear NAND bandwidth advantage over the more client-oriented designs. That said, the controller is also much more power hungry and the 1.2TB SSD 750 consumes over 20W under load, so you won't be seeing an M.2 variant with this controller."
[0]: http://anandtech.com/show/9090/intel-ssd-750-pcie-ssd-review...
http://www.samsung.com/global/business/semiconductor/product... http://www.pcworld.com/article/2866912/samsungs-ludicrously-...
SATA drives connect to the host system over a SATA PHY link to a SATA HBA that itself is connected to the host via PCIe. The OS uses AHCI to talk to the HBA to pass ATA commands to the drive(s).
PCIe SSDs that don't use NVMe exist, and work by unifying the drive and the HBA. This removes the speed limitation of the SATA PHY, but doesn't change anything else. The OS can't even directly know that there's no SATA link behind the HBA; it can only observe the higher speeds and 1:1 mapping of HBAs to drives. Some PCIe SSDs have been implemented using a RAID HBA, so the speed limitation has been circumvented by having multiple SATA links internally, presented to the OS as a single drive.
NVMe standardizes a new protocol that operates over PCIe, where the HBA is permanently part of the drive, and there's a new command set to replace ATA. New drivers are needed, and NVMe removes many bottlenecks and limitations of the AHCI+ATA protocol stack.
PCIe is packet level, NVMe is a block device interface.
In comparison, PCIe is a much more sophisticated serial protocol. It's an entire damned packet network with addresses, subnets (so to speak), routing, retry, a credit system for bandwidth sharing. Unlike ethernet, which typically tops out at achieving ~75% of its theoretical bandwidth, PCIe typically tops out at ~95% of its theoretical bandwidth. Crazy stuff.
I'd bet good money that the PCIe PHY can beat the crap out of the SATA PHY on energy/bit and that the disparity will only widen with time.