I'm not the OP, but in 2003 iTunes was the single biggest drawback to switching to the Mac, for me. I had a fairly large collection of MP3s at the time -- maybe a coupla thousand -- and a bunch of versions of the same song in some cases (an album version, a radio version, live at here, live at there), and it was all neatly categorized in the filesystem, band/album/name, when I switched from Gentoo to OS X. I freely admit I should have had a backup.

When I "tried" iTunes, there was an option for it to rename all your files (I don't remember whether it was the default or if I stumbled on it), but what I did NOT expect was that all my carefully named folders and files would be lumped into "Unknown Artist"/"Unknown Album"/"Unknown Track NNN".mp3, while my old files were all carefully removed.

My MP3 collection literally never recovered. I could choose to play the few hundred tracks that had happened to have good ID3 information, but all the tracks I'd ripped myself were essentially randomized. I also never found an ID3 creator that could successfully differentiate the various versions of songs I had, though I stopped trying around 2006 or 2007, and now I don't bother buying music any more -- I just listen to Pandora.

Poor platform support. No Linux version at all, and their Windows version is lame.

Poor format support. I don't care either way about Ogg, but no FLAC support is a deal killer. There's a FLAC plugin, but it requires using 32-bit iTunes, and even then it's not supported very well.

It insists on "managing" my music collection itself. It's constantly renaming, moving, editing ID3 tags, and what not. I think most of those features can be turned off, but it takes forever to figure out what all the options are, and by the time you realize they need to be turned off, it's generally too late and you have to pull the altered songs from backups.

It also doesn't handle poorly tagged music very well. For example, a while back I had a two CD collection of trance music, something like "The Best Goa Trance of 2010". The MP3s were tagged with the artist and song names, but no album name, presumably because they were originally on different albums. In iTunes there's no way (that I know of) to group the songs in my library, except going through and manually adding the album tags. This is made even worse by the default settings where it moves around and renames the files without telling you.

I'm sure I could train myself to immediately fix all of my music's ID3 tags, and to conform to Apple's arbitrary way of managing music, but it's easier to just use a different player.

iTunes is slow and buggy, especially on Windows. I used to be able to press Esc to shrink from a movie from full-screen to windowed, but after an update, it stops the video. Then if I restart it, it runs in full screen mode again which is quite surprising. If iTunes ever crashes or is quit, it won't remember your place in any of your files. Syncing iPods is a pain, especially when you try to find which podcasts you're syncing. And there's no way to get songs from your iPod onto your computer.