I don't think I've ever forgotten about a biker I've passed, but I also like to avoid situations where I might forget, which put the biker in a position to surprise me, or which impede my maneuverability if something else happens.
But while I'm okay poking along for a block, if there's traffic behind me then that could result in an impatient driver overtaking both of us, which could be far worse. It's a tricky balancing act.