In particular, this idea that giving a negative connotation to the color black and darkness in general, and a positive connotation to the color white and lightness in general, implies some kid of racial bias is absurd, ahistorical, and frankly disturbing (since it creates controversy retroactively out of nothing).
It's not even that I mind adjusting my language if people object to it; but there's a world of difference between "I really don't like it when you use the phrase [...]" and "you're being racist when you use the phrase [...]".
This is true, but also probably an indication that the term "whitelist" isn't as universally understood [by developers] as the "how dare you suggest using allowlist!" crowd suggest...