That depends on how bad your internet connection varies. Skype compression artifacts can be brutal. But yes, you are correct in that the microphone makes a big difference as well as the room you record in.
edit: grammar
That's the primary problem with using Skype for podcasting: you have no control over codec selection. When Skype detects network disruptions, it automatically changes codec to reduce bandwidth usage. Granted, if your network cant transport at the higher quality codec, you're going to get drops in audio anyway. Personally, I'd prefer the control though. It's easier to wait out a brief disruption in bandwidth than it is to coax Skype back up to a better quality codec.