Unless you think that every single child was previously slacking off and is capable of effectively working double time without getting overwhelmed, then it's still a problem to be concerned about.
I actually do think this.
"John Stuart was educated by his father, with the advice and assistance of Jeremy Bentham and Francis Place. He was given an extremely rigorous upbringing, and was deliberately shielded from association with children his own age other than his siblings. His father, a follower of Bentham and an adherent of associationism, had as his explicit aim to create a genius intellect that would carry on the cause of utilitarianism and its implementation after he and Bentham had died." https://en.wikipedia.org/wiki/John_Stuart_Mill#Biography
I realize that this is not financially realistic for application to every child. But the gap between how far we could push children and how far we do push children is often massive. We've always been "ok" with leaving many children behind.