If you warm up a few rooms to comfortable then it's hard to imagine the rest of the house being at freezing temperature. It's going to be some sort of gradient. But sure, this is a concern/consideration.