These tasks are all noisy, all pretty messy, but the interesting part here is not that performance improves on these constructed tasks -- NLP researchers have been improving performance on constructed tasks for decades, by customizing models and curating datasets.
Rather, the interesting part is that performance improves despite no specific model or data shifts except for scale.
It seems that you're arguing two things: first, that these tasks aren't that interesting -- on that we agree. But, second, that it's obvious that performance would improve on arbitrary tasks because of memorization and an unknown training set -- but this is easy to validate: make up a new constructed task, and see how it performs.