Imagine if everybody switched to EVs right now, en masse. Emissions over the next decade, and every subsequent decade, would be massively lower. Waiting for every gas car to reach end of life before switching is always going to be higher emissions, always.
Similarly, the "waste" already happened when the gas heater was manufactured. There's no additional waste when it's decommissioned. It's a sunk cost, there's no getting that back. The only question is if you switch to lower emissions now, or you switch to lower emissions later.
Now, if you bring money into it, sure, there could be a financial motivation to keep emitting higher amounts of emissions. But if you take monetary considerations out of it, it's always better to stop emitting sooner rather than later.
I'd love to have some serious push back against this. The best I've ever got is "that doesn't sound right..." without any engagement with the quantitation or the ideas. Which is exactly what I would expect if it was a fallacy.