It's not that Amazon is a startup itself (what, 100k+ employees worldwide doesn't count?) but that when you have a big company like that you often see people leave to make startups. Young guys spend a couple years being mentored and becoming great developers, then leave to go pursue their passions.
I have two former managers that are both very senior people in local Toronto startups, plus a number of friends who've left to hit up the start-up scene both as founders and early hires.