Yes I think you can sell the software but you must make the source available. I'm not sure how this is different from charging for distribution, from a practical standpoint.
If you are careful with who you distribute the binary to, you could in theory keep the source sort of "secret", though I don't think it's a good general approach to GPL software.