Completely anecdotal of course, but I've never met someone who smoked pot in their teen years who wasn't a layabout stoner with no future prospects in the 20s. Meanwhile, I know plenty who started smoking pot in college who were just fine.
I think any drug that affects a developing brain can have grave consequences. I would love to see research that follows people who start MJ use at different stages of their lives.