Although PHP suffers from this, I'm not sure it is alone. I suffer a lot of pain setting up Ruby environments (I don't think I've ever managed it the same way twice) and getting everything to play nice. Even Python, a language I am about as familiar with as PHP, has it's annoyances on this front.
I suspect the difference is familiarity; I can deploy a working PHP environment in my sleep, because I've done it so many times.
The other day I had to set up Java/Tomcat - literally the worst experience of my life...
Also, PHP is not helpful in local development. I think there's a built-in runserver in the newest versions, but for all of history, to develop PHP you had to install (and set up) apache and all your required modules locally.
You're thinking like a seasoned pro :) For a newbie there is xampp - a familiar looking windows installer configures the Apache/PHP/MySQL (and others) stack, then it disappears into the backround and, largely, can be forgotten.
This is huge for a new developer.
Think about it this way; if you had a class called "Learn to Write Websites", for absolute beginners, I argue PHP is a good initial choice. Because you don't have to worry about the environment, how to server it, command line, package management - which, say, Ruby would require. You just chuck in a brief overview of how servers work in general, and then get them working on some PHP scripts, with instant results.
I love this!
I like Ruby, of course - it's powerful, has modern standards, and Rails is brilliant. But for a beginner, and even moderate, developer it is a bit of a pain to work with.
With python and Django, I use virtualenv+pip to get a consistent environment, manage.py runserver to test locally. I can pretty much start developing everywhere (my desktop, my VPS, my laptop, that same laptop after I reformat it and install some new distro to test) with 3 commands.
This sort of thing (http://pypi.python.org/pypi/virtualenv) is non-trivial for a beginner.
I'm not trying to defend PHP too much here; I like it, but it is a horrible language on many levels (as are other languages).
It also just works, especially for beginners.