This aphorism is taught to doctors in training. It's like Occam's razor for the medical profession. It's not a bad heuristic, but it necessarily leads to a somewhat terrifying error rate when diagnosing people who actually do have the rare diseases.
I'm not saying you do have a rare disease, but if there is something wrong with you and you've been to many many doctors, it seems more likely than a common disease. I had a rare type of cancer and was misdiagnosed for a year prior to correct diagnosis.
My advice: browse through http://www.rarediseases.org/search/orglist.html and see if anything stands out, then demand someone test you for it.