A single successful interbreeding incident would contribute some odd genes to the local gene pool, but wouldn't contribute much genetic diversity once it spread out through the local gene pool. Remember that the data suggests exactly 2 individuals dropped into human populations that were locally in the tens of thousands, and globally in the multiple hundreds of thousands. It would contribute some odd genes to the pool of genetic diversity, but wouldn't move the bar very much.
For those who think that species barriers are some sort of absolute barrier to what is hypothesized here, it is worth considering the case of donkeys, horses and mules. Donkeys and horses are estimated to have diverged a couple of million years ago, but routinely can breed. The result are mules, of which over time we've bred tens of millions.. Mules are almost invariably infertile. However according to http://en.wikipedia.org/wiki/Mule#Fertility there have been more than 60 documented cases of female mules successfully bred by stallions. The foals would be mostly horse, and the result of that is an animal that is 3/4 horse which therefore is likely to be able to breed with other horses.
It is not unreasonably by comparison that there could be a constant level of interbreeding between us and Neanderthals, with many infertile offspring, and by chance on 2 occasions fertile offspring who have descendants today. (The overall number of crossbreeds would be lower than the number of mules, but their odds of fertility would be much higher than for mules.)