It is common to use long context embedding models as a feature extractor for classification models.