The hardware bus was async (!!!), meaning you had to arrange correct clock synchronization for every one of your peripherals independently. (c.f. "DTACK grounded", the classic 'zine of the 68k hobbyist).
The exception handling was insanely complicated, and even then they got it wrong in the first version: faulting instructions couldn't be restarted, making the use of an MMU impossible even in principle. They fixed this (via dumping even more undocumented complexity onto the stack) in the 68010, which was essentially a "patch release" to the part that was needed by the Unix workstation community.
It's true that starting the PC with a 32 bit flat address space would have saved headaches in the late 80's, and it's likely that the feature set in Win95 would have arrived a few years earlier. But on balance the mac and unix worlds had not less grief moving off of 68k.