You're wrong. Remember that XOR works on individual bits. If what you're saying was true, then swapping bits that are both set to 1 would also fail, which means this algorithm wouldn't work at all.
Edit: Ignore this post, I misread the original post as saying swapping the same values would fail.