Although different this reminds me of those (even those close to you) that voice their thoughts that working on a startup and not working a 9-5 job is less respectful. I file these under the "Wh-what?" replies.
I think, deep down, people know they have the ability to do anything they want, but they are scared to let the thought even bubble up through their subconscious. Either that or they're damn lazy. Its cool, but I resent the whole "luck" thing.. the more I work hard, the more I realise hard work is the only way to succeed.
I think the startup world is more random, though, and hard work only gets you in a position to take advantage of opportunities, as someone mentioned elsewhere regarding "The Black Swan".