I do think your point that the physical repetitiveness isn't the only factor is correct, however I believe it is the main driver.
Almost all physically repetitive exercises combine sensory and cognitive tasks to varying extents.
I would also argue that learning and implementing new movements is a much more diverse task cognitively than getting better at beatsabre.