It's already a thing [1]. They also have a project website [2] with some nice videos, although the code hasn't yet been released.

[1] https://arxiv.org/abs/2308.09713

[2] https://dynamic3dgaussians.github.io/