It's not a matter of "It's hard to keep up with the brilliant folks". It's that they are often actively destructive to the productivity of the team. Hence my co-founder's comment.

Brilliant people can often be easygoing and a great benefit to their teams. They do it by moving the whole team forward. They're never 10x individually in that case, but they may elevate a 3x team to 5x.

edit: Find me an engineer who is both brilliant and humble, and I'll bend over backwards to hire that person.