The costs are totally prohibitive, though. We're talking tens of thousands of pounds, whatever direction you decide to take.

You could spend 10s of thousands of pounds in a "properly sized" heat pump system. Or you could spend 10s of thousands of pounds in insulating your home + a more moderate heat pump.