there is no evidence that they were near to each other for 50K years, only that their genomes diverged 50k years ago which was the last time they were near each other.
At the separate-times-and-places that they died in "France", the recent migrants may only have arrived in the last few months, having spent the previous 50K years over in that other far away place, and died only somewhat near each other a decade apart
the mixed genome of early modern humans and the lack of mixing of Neanderthals is a separate piece of evidence, and it may point to Neanderthals not being as sexy-social, but that's just a "may".