Hacker News Reader

This is an interesting problem to solve. For the sake of better understanding, can OP or someone else here suggest research papers or code that describes similar approaches to detect and remove outlier data by analyzing the embedding space.

View on HN

In the example shown in the product tour, we use an approach based on Diversity Sampling. Basically we look for the datapoints that would be the most representative of the drifted domain (and therefore be outliers to the training domain). Here is a blog post that summarizes some of those techniques [1]. I also found this paper that describes a similar approach [2]. Happy to chat more about it.

[1]: https://towardsdatascience.com/https-towardsdatascience-com-... [2]: https://arxiv.org/abs/1904.03122

Thank you. Will take a more closer look and reach out

Reply on news.ycombinator.com