It's not really a question of the "correct" data. To evaluate whether an observation is anomalous, you typically define a threshold for what's considered not-normal, e.g. an observation 2 or 3 standard deviations above the mean.

This may leave you with some false positives (like any system of this nature would). Of course, you could go the route of actually defining an anomaly yourself and building a more sophisticated model (i.e. one not using z-scores as the measure of anomaly), but that's obviously a different scope.