Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. If the goal is AGI this might get exponentially worse. Intelligence needs to take resource consumption into account. The models we produce aren't even close to high level reasoning and we're already consuming significantly more energy than humans or animals, something is wrong.
The scale argument isn't great either because deep learning is running into the inverse issue of classical AI. Now instead of having to program all logic explicitly we have to formulate every individual problem as training data. This doesn't scale either. If an AI gets attacked by a wild animal the solution can't be to first produce 10k pictures of mauled victims, intelligence includes to reason about things in the abscence of data. We can't have autonomous cars constantly running into things until we provide huge amounts of data for every problem, this does not scale either.
That is something of an illusion.
Obviously there will be some sort of uneven distribution of computing power; some institutions will have more, some less. The institutions with more power will create models at the limit of what they can do, because that is the best use of their power.
So if the thesis of more power = more results holds then truly cutting results will always be by people with resources that are practically unattainable by everyone else. Google's AlphaGo wasn't a particularly clever model, for example. It just had a lot of horsepower behind it to train it and the various ranging shot attempts Deepmind would have gone through. Someone else would have figured it out albeit more slowly in a few years as computing power became available.
Computational power is still getting exponentially more affordable [0]. Costs aren't really rising, so much as the people who have spent more money get a few years ahead of everyone else and can preview what is about to become cheap.
[0] https://aiimpacts.org/recent-trend-in-the-cost-of-computing/
Most of the article is describing past scenarios, only the last 3 paragraphs make the argument that the past is a good representation of the present
https://twitter.com/ylecun/status/1066568396177842176
i.e. gradient-based learning is the final word on the matter.
_
/ \
,----------------------------.
| Hey! I found your minimum! |
'------------ ------------' /
.----. \ / / \ /
/ \ \/ / \ /
/ \ @ / \ /
/ `'--' \ /
/ \ /
/ \ /
/ \ /
/ \ .-. /
/ \_/ `\ /
\.__.' \ /
\ /
`;._.-'