First, most athletes do train every day. Even tapering before an important competition involves training daily. I ran track and XC competitively in high school and college. Some of my teammates hadn't missed a day in years. Most of us only skipped training if we were injured or very ill.
Second, the brain is not a muscle. Sleep deprivation and stress hormones can diminish its abilities, but no amount of thinking or drudgery can damage it. Neurons can't tell if they're reading, playing video games, or debugging code.
An examination of history also supports this. A mere century ago, people toiled far greater for far longer than we do today. The average work week was over 60 hours![1] Despite such exertions, even the concept of burnout didn't exist. It still doesn't exist in some cultures, and in the cultures it does, burnout doesn't affect everyone. Unlike sleep deprivation or physical exhaustion, some people just don't burn out. If you start looking at studies, you'll find papers like, Is burnout separable from depression in cluster analysis? A longitudinal study.[2] The answer: no. So far, researchers haven't been able to reliably differentiate between burnout and depression. This indicates that burnout is depression, not a guaranteed consequence of working.
Of course, one should still be wary of depression, and work is one piece of that puzzle. Yet work alone is neither necessary nor sufficient to cause ruin. Exercise, social gatherings, and friendships are equally (if not more) important.
As one more piece of evidence that work alone doesn't cause burnout, I submit myself. I have written code every day for the past 908 days.[3] That's almost 8% of my existence. Those commits aren't just whitespace or linter changes. Every day, I write real code that runs in production. I also exercise, socialize, and generally live a fulfilling life. People sometimes voice their concerns about burnout. But to me, it's as if they'd asked, "Are you worried about getting burnt out from reading so much Hacker News?"
1. https://en.wikipedia.org/wiki/Working_time#Gradual_decrease_...
2. http://geoff.greer.fm/files/Bianchi_20R__20et_20al__20_28in_...