I feel like it's almost a requirement to take this opportunity to share Technology Connections great video [1] on the topic.
Personally, I've used a heat pump as the sole heating unit for my house for the past 6 years or so. There have been a few cases where I've opted to use an extra blanket, but overall the results have been pretty impressive, and the cost savings are pretty huge. (Apartment, so no fuel heat, it's electric or nothing)