It's also amazing what a "sorry, x,y, and z are very hard for me, I'm doing my best and still learning to do better" will go. Of course, not everyone receives that with grace, but that speaks to their character.
And for my side, I try to distinguish between someone being a jerk and someone I just don't mesh with. I'm thinking of a specific coworker right now. They and I mixed like oil and water. They're smart, competent, and well-meaning. I respect them professionally. However, something about them rubbed me the wrong way, and I'm certain they'd say the same about me. We were never going to be work buddies. They weren't being a jerk, though. Our personalities weren't compatible, no more. I'd still give them a good reference.