Very interesting!
I’ve got an internal tool that we use. It doesn’t do the deterministic classifier, but purely offloads to an LLM. Certain models achieve a 100% coverage with adversarial input which is very cool.
I’m gonna have a look at that deterministic engine of yours, that could potentially speed things up!