In situations like this (for example if the women who mentioned having a boyfriend had been more direct "please stop trying to spend time with me") it would cause its own set of problems. An intentional harasser will insist nothing is wrong and keep coming back... at best. Some cause problems at work or become verbally/physically abusive. OP may have awkwardly stumbled into this situation, but his colleague doesn't know how to stop it without everyone getting pissed off. Is it worth HR involvement? Depends on what's been said. But it's not like they would understand each other from one conversation.

This took time for me to understand. I had an acquaintance starting at a company who was interested in my separate traveling job. When I moved to SF she said we should have dinner, we were cool, invited to her birthday. She suggested I find a souvenir for her on my next trip. Months later I gave her a nice scarf and never saw her again. She probably forgot that it was her idea! I asked her to meet up, and she'll agree, but then it never happens. If I say, "hey, why don't you tell me why we didn't meet up? I need to talk this over in person, you need to believe me", that's exactly what a harasser would do. So I've had to let this go. It might be a little uncomfortable, but no one really gains anything from fixing it, except trying to feel like I'm always right.