I think you've hit the nub of the problem with your statement that you're "more than 99% sure CFS is a physical condition". The majority of people (including many doctors) simply don't understand certain basic facts about how the body works (such as the fact that cortisol is a stress hormone, but also one of the main modulators of the immune system). Most people still have a Cartesian/Victorian view of the human body, and believe that the mind and body are completely separate.
Telling someone to "get a new hair cut to cheer you up" won't be much use for major depression or PTSD either. Comments like this just perpetuate the misconception/prejudice that psychiatric conditions aren't real illnesses. I think it's a combination of this prejudice, combined with a misunderstanding of how the body works, that has caused most of the problems relating to understanding CFS.