I find it particularly interesting because you'd expect OSS that will be picked up by others to have good code quality though. Add to that the fact that at the time your typical PHP user probably had more technical competence than most (webhosting, particularly supporting PHP was rarer then)
However, it does some things that keep me up at night. It stores serialized PHP arrays to define what users have access to (the MU branch). It strips away magic quotes based on runtime configuration, only to add them right back using a PHP loop. It does not use any form of output buffering. It's themes are PHP files that often times contain core business logic (somewhat of a PHP problem really, but still). WordPress is bad code and there is nothing anybody can do about it.
This of course is not true of all OSS projects: far from it. Just so happens that when a small community controls core development and the initial codebase is not great, which is the case with many early PHP projects, the outcome is that the code will stay bad.
When I say that the code inside of WP is awful, I mean that you should not look at it and you should not try to change it. Don't try to make WP into something it's not, like running custom applications on top of it, etc. When somebody tells you they want to integrate something with WordPress: run. However, I use it as a simple web publishing platform and it gets the job done.