In a few months (weeks, days - maybe it has already happened) models will have much better performance on this test.

Not because of actual increased “intelligence” but because the test would be included in model’s training data - either directly or indirectly where model developers “tune” their model to give better performance on this particular attention driving test.