You're right in that it can be said about
almost anything, but it's not about time. If you're born into poverty, you can have all the time in the world, but you'll find it extremely hard to get proper education before you even get a chance to put some time into it. In other words, to even get a chance to commit time to something like mathematics at the level we're talking about, there are hard practical social and economic preconditions which have nothing to do with one's abilities, or even the field of endeavor as such.
The article just conflates the field itself with these wider socio-economic conditions, constructs a weird argument out of the confusion, and comes out really bad.