By refactoring what I mean is continually revisiting the architecture of your code, identifying common functionality, better organisation of code, better abstractions. The right decisions for your codebase change as it evolves and so you need to keep reexamining these (implicit) decisions.
Continuously incrementally refactoring your code enables so many things to work better. Your example here is a great one - don't implement something you don't have a direct need for. Don't design for it. If you have a culture of refactoring then you can be confident that in the future if that need crops up, you can implement it with appropriate refactoring that the result will look at least as good as if you designed it in up front. If you don't refactor as a team, then you have to do it now, or putting it in later will result in a mess.