And even those will require draft from outside the insert / stove, so either you build them with an inlet from outside, you pull cold air from badly sealed doors / windows, or the chimney draft will be insufficient.
As a kid I lived in homes with both - and a home with a barrel stove - and as an adult with a pellet stove and I don’t remember that being a problem. Net, it was fine?