Hey this is really cool. Do you mind sharing a little bit about your approach? Is it similar SVM like algorithm or something else entirely?
The model appears to be similar to MiniLM-L6 (384 dimension, 6 transformer layers) but uses RoBERTa/GPT-2 style embedding/tokenization (50265 vocab size), used for binary classification, and quantized to INT8.
My guess is that is was distilled from roberta-base, then fine-tuned on freely available pile like artem9k/ai-text-detection-pile or similar.
It's a nice model - I've just used it to create a browser extension which highlights text based on how likely it is to be LLM genereated.
Edit - a quick google search reveals ibm-granite/granite-embedding-30m-english with the same architecture: slap a binary classification head on, fine tune, job done.
> MiniLMv2 L6-H384 (30M params) progressively distilled L24→L12→L6 from RoBERTa-large-v8 teacher (neobert-v2 recipe, min_words=0). Byte-level BPE tokenizer. INT8 quantized.