Do people actually think you are unemployed, or do they just think that what you are doing cannot possibly support you full-time/comfortably?
I experience the second reaction fairly commonly. People ask me what my job is, I explain it to them, and then they inevitably follow-up with "Is that profitable?" or "You work on that full-time?" -- basically, some variant of "that sounds like a nice hobby, kid, but what do you really do?"
I think it stems partly from insecurity, and partly from lack of understanding.
The insecurity part comes from the fact that most people don't want to actually believe that you can be successful by working for yourself. Many people dream about working for themselves or starting their own company, but few are willing to take the actual risk. When they see other people fail, it confirms that their decision to take the "safe" path was the correct one, and makes them feel better about themselves. This may seem really cynical, but I've experienced the reaction enough to believe it. Felix Dennis, founder of Maxim magazine and a publishing empire, also talks about this behavior a lot in his book "How To Get Rich" (tacky title, great book).
You mention that you're not a materialist. In that case, does it really matter if other people don't take you seriously, or fail to give you the respect you might deserve? Presumably, your friends should understand enough about your situation and business not to make such job offers. And if it's someone that you have to impress by handing out a fancy business card or driving a fancy car, and if they won't take you seriously without one, is that someone you want to be associating with anyway?