Even if you don't ever mix the 9-5 time with the freelance time, (Which FYI is nearly impossible in practice.) you just aren't going to be at your best when you show up the next day to work your 9-5 8-hour day after having worked a full day and moonlighted for another 4-6 hours afterward the day before. You're lying to yourself if that's what you think. And if you're working weekends, you're going to get burned out in a way that only punishes your 9-5 employer for not giving you a higher workload. People get fired all the time when their weekend or after work binge drinking interferes with their work the next day, why should spending your discretionary time burning yourself out be any different?
Human beings can only do one thing in any moment. Worse, this truth has some applicability over longer time scales as well.
Human beings require substantial periods of recovery in order to perform optimally. Often, what you do during that recovery has dramatic implications on how well you're going to do when you get back at it. (Ever solve a difficult programming problem only after taking a break? You won't get those same eureka moments if on your breaks your loading entirely new problems into your brain all the time.)
You do the math.