I completely agree with you. There's never a true "rule" of when to refactor. The 1, 2, n pattern or as I've learned tonight, "rule of three" is more of a basic guideline than it is a steadfast rule.

I think it's easy to say, "If you copied a piece of code three times, please put that in a common place" to a junior engineer as a basic lesson for getting started with refactoring.