I'm a little insulted that people still consider it a disease. It's like saying people that don't like a particular food have a disease because most people like the food. It's quite simply not the case.
If a person disliked the flavor of all foods except one or two, would that be a disease?
Given the serious negative impact this difference can have on a person's life, yes, it should be considered a disease.
IMO, the best analogy is closer to psoriasis. It's believed to be at least partly genetic, a lot of people have it, but only a few are totally debilitated. Since for the majority of sufferers, it only amounts to a little extra lotion in the morning, should it be considered a disease or a difference? I'm sure the people with mild cases would like even that to be gone.
In other words, do you dislike the use of the word "disease" to describe that something is missing compared to the "standard human", or do you disagree that there's anything bad about being deaf or autistic?