I find this even with activities that require novel dexterity. (e.g. learning something on piano, or something in leather/woodwork).
I get a bit better at them immediately, but that plateaus. (Often, I actually start to get worse at it, as I fatigue.)
On returning after a break I'm usually much better at it than my previous best.