When Good Patterns Go Bad: The False Positive Regex
blog.donbowman.ca
blog.donbowman.ca
The real owner may be able to search for a few things,like the last 4 of their credit cards, or prescription names, or their social, and find and remove things, while an attacker would have a much harder time sorting through the data.
It doesn't really affect your main point though. Bad regexes and the fight to correctly classify items is always a tug of war between correct and pretty.
And false negatives (missed info) aren't much better either: sensitive data often lives in images, as people's profile photos or passport or ID scans, where you must really look at the pixels (OCR rarely helps). Here regexps instafail because they assume input is text.
Personal war story: I encountered a bizarre category, sensitive data that's true negative which people nevertheless perceive as true positive. Think "specimen (invalid) passports" or "sample (invalid) credit cards". Not really sensitive (zero risk), but people still expect to see them detected. Creates some funny situations during testing. If you're curious, I wrote about it here [0], with some examples.
so maybe we need an EICAR for PII?
…not great for sensitive data detection in general, of course. Hopefully clear from the context (ha!).