That is the reason you check the freelist in the unlikely case that you see it, the cost would average to near nothing except for the memory accesses to set, check and clear the value.
You could also xor the memory location into the marker to avoid multiple collisions for applications that allocate a large amount of similar objects.