I had a slightly different problem - once I left and made the odd appearance in casual clothes, for some reason it was assumed I had become a highly paid consultant of some kind, living "the life" and that I "had all the luck".
I think, deep down, people know they have the ability to do anything they want, but they are scared to let the thought even bubble up through their subconscious. Either that or they're damn lazy. Its cool, but I resent the whole "luck" thing.. the more I work hard, the more I realise hard work is the only way to succeed.