- (Correct) IO is ridiculously non-portable and painful in so many ways that it isn't even funny anymore
- Locks are ridiculously non-portable and painful to the point where you're better off just using "mkdir"
- POSIX is stuck in the "everything is bytes and we slap an encoding on it some of the time" era thinking. This makes it painful and hard to implement proper text handling in many instances. This also leads to a lot of bad behaviours.
- Memory management is IMHO lacking from a user space perspective. For example, it's practically impossible to implement a cooperative memory cache on this. To the best of my knowledge no OS has the necessary interfaces, though.
- ioctl as you mentioned
- SysV/POSIX IPC is so bad that no one ever bothered actually using it for anything
- Personally I think it's a misleading API (conceptually, see above, the text example for example) almost to the point of deceptiveness. It's very easy to write correct looking programs that behave far from intended, especially in edge cases. IMHO code using it is practically unreviewable in everything but the most trivial cases. Non-portability is practically guaranteed, you have to test every platform. Portable code usually turns out to be quite ugly due to platform deficiencies and minor API incompatibilities.