In the first instance, the person in question was my direct boss and was responsible for reviewing my code. He'd routinely call me an idiot, a retard, etc., and during meetings he'd often say things like "no, we can't give that to Jacob - he'll just fuck it up." Now, in his defense I was a pretty shitty developer at the time, but I do believe there are more constructive ways of reviewing code than calling the author a retard. I dealt with it for a few months, and finally asked his boss to transfer one of us. When I explained the reasons, this person was fired.
In the second case, my coworker was showing up to work drunk, yelling and cursing at coworkers, and routinely threatened violence -- "I'm gonna beat your ass if you don't shut the fuck up" was, I believe, a common line. Once he threw a chair through a window halfway through a (technical) meeting. Again I was too much of a coward to call him on his behavior directly, but yes I spoke to first my boss, then his, then the owner of the company. Nothing changed, so ultimately I quit.
In retrospect I certainly didn't behave perfectly: I should have confronted the behavior directly, and I should have made clear what my expectations were with regard to a professional working environment. But I was young, and all in all I'm proud of myself for not tolerating abuse (even if I didn't go about it quite right).