This was my experience as well. When I see geniuses that coast through everything thrown at them, I wonder if I have any advantage over them in the future, having struggled before, or they're just so smart that nothing touches them.
I think these people are still working hard in a sense, they just have a low tolerance for work that doesn't interest them.