But then they're effectively excluding the best-interviewing groups from the random experiment.
Can't you pick the best interviews, then hold your experiment with random groups as long as you mix in a the data from the interview groups in such a way to build a representative sample?