I used sentence transformers for clustering for a similar use case: https://huggingface.co/sentence-transformers