This might sound rude, but people remember things they care about. Maybe you legitimately don't care about all these names as much as you think you ought, and feel guilty about it?

I've given up and am honest with myself about how many human relationships are transient and not worth holding onto.