In some Asian vernacular Englishes (e.g. Indian, Singaporean), this is means "yes, right" / "I agree". I used to experience your reaction to people saying "correct" to me while I talk (i.e. "fuck you") but since then, my understanding has increased, and I interpret it to mean "yes, right" / "I agree".
Additionally, it may be that people who say this aren't native speakers of an Asian vernacular English, but are accommodated (i.e. Linguistic Accommodation) to it and so reproduce it in other contexts (e.g. in conversations with you).
I hope you found this illuminating.