I disagree, there are a lot of applications in IT forensics. I have seen a masters thesis about the analysis of instant messenger texts. To identify illegal material with ML this content is essential.
That's good input, thanks. I had submitted a reply as a question, but now wonder if I should rephrase it as a statement to be on the safe side:
It could be very important for the containment (for lack of a better term; I'm not familiar with the area) and pausing/deactivation of those models to be well-tested and effective.
(in the same way that machine learning may exhibit unconscious bias based on the datasets it is trained on, I fear that dark psychological traits could potentially be learned from traumatic content)
How do you propose to enforce this? (for human or machine)
Discussed in practical, psychological, and philosophical terms verbally and textually, any determinations written legally, ratified and endorsed internationally, and enforced using existing jurisdictional mechanisms. All of those are broadly human factors, although machine-based detection, discovery and tracing could certainly be used to provide multiplying support.