It's not always interpolating the sounds that would actually occur. Yes, a hammer hitting something will produce the relevant sound in my head, but it can be quite a bit more abstract than that.
Imagine someone dancing in an animated gif. Each motion has a particular "swoosh" or "pop" or "bang" to it. It's kind of weird to describe. As the gif replaces, the same sounds play over and over. I couldn't turn it off if I wanted to.
Take this for example: https://i.imgur.com/eVeYyCq.gifv. In real life I imagine I would hear the woman grunting or growling as she tried to lift the tire. But in my head I can only hear the motion of her legs as they shift to accomodate the weight, making sort of a "kathoomp thoomp thoomp" each time her feet touch the ground.
Makes me think of how a musical score can capture the energy of a scene so well; I'll bet those musicians can probably "hear" the scene without even trying.