Always interrupt if you have something constructive to say in a casual/business conversation (unless you're in a debate), that's how you gain trust and credibility.
Very occasionally, the right response is to interrupt them impolitely. Much more often the best thing is to let them finish. It's an almost passive-aggressive solution, but sometimes it's the best you can do.
(you can ban me now. I won't regret this)
This is dangerous advice. Being perceived as someone who frequently interrupts is socially detrimental. As another commenter said, save it for when someone really makes it necessary, and converse with grace at all other times.
People label others with hostile things like "know-it-all" as a way of dealing with their own inferiority complexes.