It's a small expense for employers, and can be a real difference in costs for the employee. I don't think you're correct. If I spend 100% more on heating everyday because I work from home so need to heat 24/7, that's nothing to an employer (100-200$/m) but could be a significant deduction from a family's budget.

Meanwhile the employer would have otherwise heated up an office space and paid all bills there.

I've been remote for a decade now and it always struck me as odd, but I didn't care enough to push for it. But as the remote concept is being used by many employers to cut on office expenses, employees with no alternative should be compensated for the extra expense they incur. It's just common sense, like paying for your mileage when you drive for business.