Having watched the video of what he accomplished on the C64 I am now retroactively furious at every drop of framerate I have experienced on every platform ever.
Interestingly, in a modern environment (unlike in the 80s-00s), both of these seem like choices that could be made either way.
Re #1 — all the major OS manufacturers (Microsoft, Apple, Google) now have their own flagship devices running their OS, and could standardize their app-store certification processes to involve perf testing on their own hardware if they wanted. Especially for the mobile hardware: there's no reason the QA process for e.g. releasing a game on iOS can't involve certifying zero stutter on a given iPhone, and then restricting the game to only be playable on that iPhone or newer (the way consoles effectively work right now.)
Re #2 — there's no reason, other than inertia, that personal computer (including mobile) OSes still put every single task on the system into one big (usually oversubscribed, from a realtime perspective) scheduling bag. We could—using the hardware hypervisor built into all modern architectures—split PC OSes into two virtual machines: one for the foreground "app", and one for everything else; and give the foreground-app VM fixed minimum allocations of all the computer's resources. Then you could make real guarantees† about an app or game's performance, as long as it was said foreground app. It'd be very similar to the way some game consoles (e.g. the Wii) have separate "application processors" and "OS service processors", to ensure nothing steals time from the application.
† And those guarantees would also be requirements: if the foreground app says it needs its VM to have 5GB of RAM, you literally won't be able to run it if you don't have 5GB of free physical memory to hand it (though the OS would probably first try to OOM-kill some sleeping apps to give it that memory, like iOS does.) Much clearer than the current "this game will be really slow if your computer is more than four years old, but it's not exactly clear which part of the computer is below-spec" we have today.
To guarantee that, testing has to go through _all_ possible game states. For almost any non-trivial game, that's infeasible.
"and then restricting the game to only be playable on that iPhone or newer"
There is no guarantee that newer hardware would be faster for every possible program execution, and even if it were, timing differences could affect game play.
There also is no guarantee that newer hardware produces the exact same results. For example, better anti-aliasing or fonts drawn at double resolution could affect hit detection.
This even isn't guaranteed on the 'same' hardware. For example, there might be C64's that don't have the bugs that this demo exploits.
So right there on the store website is the "demo video".
For an historical analogue: you can show hours of paying PacMan without finding its kill screen (http://www.donhodges.com/how_high_can_you_get2.htm)
The video doesn't guarantee bug-free play, but it can give us confidence that the benchmarks are an honest test since we can see what the test looked like.
If it's a text editor and it just shows that opening a new blank file and typing a few words is fast, or a game that shows the game runs well in an empty room level, then we know the demo is dishonest. And having a "boring" video in their store page won't win them much buyers. You want all the ad content to look as good as possible, so the demo video would be best if it shows off the most impressive features of your application, which is where we'd expect performance problems.
That said, it's probably not feasible because creating a full benchmark script that also doubles as a real-world demo for an app would be too much burden to put on developers.