Heck, maybe they would - they're a genius now.
Do 10 min of pushups for 6 weeks and see for yourself.
You can also do 5min of pushups, 5min situps can't drop much beyond that but still unless you are in stellar shape you'll see an improvement over 6 weeks (even 2 week).
I've seen plenty of people going from 0 pushups to 10-20 within a week by trying to do it daily for only a couple of minutes, like with most exercises the results of the first 4-8 weeks are the most noticeable.
That's of course doesn't mean that you can get into shape by spending 10min a day, but the for a single exercise that's more than enough when you spend 1-2 hours at a gym you rarely put in more than 10min per exercise unless it's aerobics.
So the brain analogy can hold because you are training your brain to do certain things, and you get better at it, it doesn't necessarily mean that you get smarted just that your brain gets more adapted to performing certain tasks. Solving sudoku is very hard at first but after several weeks of doing it daily you get quite good at solving such problems, same goes for crossword puzzles and other similar things.
P.S. On a psychological level that's one of the easiest ways to convince some one to do it because it takes nothing from them and they can see almost an immediate improvement dragging some one to the gym and forcing them to stick isn't easy, showing them that they can noticeably improve over weeks can give many people that moral boost they need to actually start investing more effort into it.
The behaviors are not the same. It is not valid to draw a post hoc conclusion from one to the other.
There is zero reason to believe "there's a better chance it is so than otherwise".
You may wish to brush up on your logic.
What I'm saying is, if you give an intervention of thirty minutes of activity a week and see no change, it's a pretty reasonable to conclude "30 minutes a week for six weeks is not an effective dose" and not jump all the way to "This intervention has no effect". This is why I highlighted how low the 'dose' of the intervention was.
[I also am relying on my understanding of Dual N-back, language learning, and other mental tasks, and the turnaround time for them is substantially longer -- and thus requires a higher dose -- than starting from a similar place in exercise. This context is useful only for general dose estimates {{ even 10 hours a week might not be enough to learn a language, certainly not in six weeks }}, but not necessary for the comparison.]
Does that address the substance of your objection?
Average couch potato could do miracles to his/her oxygen uptake with that amount of hard aerobic training. Ten minutes running uphill at full speed, stop when you're out of breath. Catch breath, do again. If you started with coopers test result of 1500m, you could probably hit 2500m at the end of the training.