You could probably use some form of DMP [0] for this. It basically learns movements by viewing them as the difference from a spring-damper system. This allows all kinds of adaptations.

[0] https://ieeexplore.ieee.org/abstract/document/5152423