Well, homophobia is generally supposed to be considered in the light of Yoda-ism "Fear leads to hate..."
As an example though, I'm not a pedophileaphobe - I'm not afraid of pedophiles. I think what they do is wrong.
The number of people who are "afraid" of homosexuals is miniscule. Most people who you'd label "homophobic" actually just believe that homosexuality is wrong, not that it's scary.
So the "homophobic" tag is pretty silly.