Modern heat pumps are cheaper than oil for heating just about everywhere. They're cheaper than natural gas in most places, unless electricity prices where you live are particularly high.
I have a heat pump btw., with COP 4.5 (below ground). Costs me EUR 2.5 - 3k per year to heat the house.
I don't really understand what the aversion is to forced air climate control here other than "it's not as comfortable" which from what I've gleaned from other people is taken to mean noise/moving air/humidity. Coming from the southern US, I find all of those points to be a non-issue for me. I've slept with a fan on my entire life, so if I can shave off 50% of my heating costs for a few decibels of fan noise, sign me up!