For engineers who are in the habit of thinking of their job and their career (and maybe a lot of other things?) exclusively in mechanistic terms of inputs and outputs, it's a tempting abstraction.
But when you work remotely, you're basically advertising your work as commodity work. You're basically consistently reminding the company -- and the job market -- that you are substitutable. In economic terms, that is simply not a smart signal to be sending.
It's also short sighted in terms of the arc of your career. Right now you might be making economic value in the marketplace strictly by extruding a software widget from your keyboard. Great! But at some point, the odds are good (not 100%, but good) that the way you'll make value is by enabling others to build software; reading body language, detecting issues in a team; or negotiating; or motivating; or selling; etc. The proportion of people reading this who will hack til their retirement day is quite small.
Sitting in your home office being "productive" isn't preparing you to do much that's important in the second half of your career. It's a case of over-fitting. You're optimizing around your current skill set and not thinking about the larger portfolio of useful -- human -- skills.