and apply a classical machine learning algorithm such as the probability calibrated SVM. This usually beats bag-of-words classifiers as it is able to suss some of the meaning of words. The advantage of this approach is that it very fast (maybe 30 seconds to reliably train a model)
It is also possible to “fine tune” a BERT family model using tools from Huggingface like so
https://huggingface.co/docs/transformers/training
my experience is that this takes more like 30 minutes to train a model but the process is not so reliable. For some tasks this performs better than the first approach but I haven’t gotten it to reliably improve on my current models for my tasks.
I am planning on fine-tuning a T5 model when I have a problem that I think it will do well on.