It worked, well, terribly. The startup folded in a year and a half mostly because of code-quality issues. I remember talking to the founders near the end of my gap year, where they said that of the 10-ish or so high schoolers they hired, only two of us really produced more than we cost (even on our piddly $20-30k/year salaries), and even then we needed a fair bit of management support.
These were also not random people - one of the founders had been a teacher at my high school (he'd been an entrepreneur with a successful exit before then), and he basically skimmed the top of the class off and got them to work for him. Many of us later had successful careers at Google, Bain, or hedge funds.
But there's only so much that you can do to compensate for inexperience. There's a lot that you learn in your first job or two, even if you already know how to program. And until you learn it, there's a good chance that you'll be negatively productive.