For sorting numbers, an algorithm is appropriate, e.g., heap sort, once we have checked the algorithm and shown that actually does sort. For outlier detection, we have no such simple criterion that the algorithm did anything meaningful.
For some first progress, we can consider the framework of statistical hypothesis testing with Type I error, Type II error, and the probability of Type I error. If we can do some derivations so that we know the probability of Type I error, then we have a meaningful way to know how seriously to take the result.
E.g., suppose we are receiving points one at a time. Our null hypothesis is that the points are independent and have the same distribution. Then Type I error is saying that a point violates the null hypothesis, is an outlier, when it does not. A Type II error is saying that the point does satisfy the null hypothesis when it does not.
Generally we want to be able to adjust, set, and know the probability of Type I error. Then we have an outlier technique with some meaning. If in addition, e.g., via the classic Neyman-Pearson result, we can show that for the probability of Type I error we have selected we get the lowest possible probability of Type II error, then we have the best possible test.
The OP is too eager to work with a Gaussian assumption. Too commonly in practice, that assumption is absurd. Instead, we should be able to work with a distribution free assumption, that is, the null hypothesis assumes that all the data has the same distribution but we don't know what that distribution is.