Context switches take a lot of time, but sometimes - like day job vs. side project - are unavoidable. So instead of trying hard to avoid all of the switches, maybe let's optimize the switching process so that it doesn't hurt so much?
I started doing that at my last work - 10 minutes before leaving the office I would open up my "Context Dump" file and write down exactly what's on my mind - current project status, what I was working on, what have I done, what has to be fixed, what quirks are there, what ideas I was considering - pretty much a stream-of-consciousness-like mind dump to text file. And then, the next morning, first thing I would do would be to open up that file and re-read it. It helped me to rebuild my mind state in few minutes instead of an hour. Also, my anxiety went down, which reduced the amount of time I procrastinated on HN each morning.
It works really well for software development: I can do the programming for a while until I can program no more, at which point I can switch over to designing interfaces. The mix between creative and analytical thinking is highly beneficial to my workflow.
When I focus on only one task, the quality of my work, at least from my point of view, is never as good.