makes me wonder if the reason why it is a lot harder to pick things up when one is older is primarily the lack of large blocks of available time as opposed to the brain being less "plastic" or adaptable as one sometimes reads.

When you read famous musicians' bios, for example, there always seems to be something along the lines of "I practiced 8 hours a day every day", hence lots of quantity over several years, but this is of course next to impossible to do in adulthood with all the demands on our time.