Lyft changes assigned drivers for reasons that have nothing to do with priority pickup. Absent a confirmation from Lyft, I’m skeptical that was the reason.
Driver switching has been happening for years though, way before I saw priority pickup as a rider.