As long as you understand the code line by line and as an overall entity, otherwise you are no better off. Just because you can read the source doesn't mean you actually can.
Even if you can not read and understand the code you are still better off with the open source. Due to the fact that independent researchers or programmers unaffiliated with developers will be able to audit the code. While with the closed source software, the only information you have about the inner workings of the software is provided by the developing company.
With open source, the chances that somebody will catch a security flaw are generally much higher. It's not necessary for each individual user to understand the code as long as some people do.
I am not the only human being on this planet, you know (And if I were I would not need skype anyway).