Is it possible that in the circumstances, they can't find their nests? I assume that normally all the birds don't fly off all at once, so maybe that makes it easier to find their nests - perhaps they remember whose nests are next to whom (or is that too complex for birds?)- but when all the birds fly off at once then it becomes impossible to find one's own nest?