Hacker News Reader

Perhaps it would be more accurate to say that marginal return is materially positive for all feasible n

A logarithmic increase in performance is positive for all n, but not necessarily _materially_ positive for all n. In fact there will be an n where the marginal gain from adding more data will _not_ be worth it, if the cost of adding more data scales linearly but the benefit scale logarithmically.

Right. If the cost of adding more training data remains flat or increases, a logarithmically increasing n will eventually reach a point such that it's more expensive to continue increasing training volume.

In fact, unless the cost of adding and using training data exponentially decreases over time, it's a mathematical certainty that a logarithmically increasing n will quickly incur expensive, diminishing returns for using more data.

So in the context of this Google paper, you could conceive of a situation where training data actually becomes easier to load (albeit subexponentially) and still becomes too expensive to use relatively quickly.

Reply on news.ycombinator.com