Yeah, sort of, but I feel like this is because most people don't really understand it's not exactly the same thing as sitting around surfing the web. Obviously this isn't a physically challenging job, but it is a mentally challenging job (and the brain is a physical organ too, remember simple physics conservation principles--none of your thoughts come for free, they all cost energy, and complex thoughts take massive energy!). That drains you in an entirely different way.
I would almost argue that having a physically tough day makes me feel sore and tired, but _accomplished_ at the end of the day. Having a mentally tough day doesn't confer any sort of sense of accomplishment, it's just a draining feeling.
Also consider the "happy" ending to Office Space--the protagonist ended up much more satisfied doing a physical construction job in the end.