This is a weird "problem" to me. I follow about 300 people (@edropple) and never feel overwhelmed. I've never used lists or filters or anything. I dip in and out of Twitter as I please. Basketball game's on? Twitter's on. Working? Twitter's off. The best analogy I can think of is that it's portable bar chatter. You go when you want to bullshit with friends, you don't when you've got other things going on.
Maybe the problem is trying to follow the entire Internet and lacking the self-control to pick and choose.