I'm trying to understand but struggling -- so it is specifically the phrase "one of the only" that bothers you, but the example of "only five minutes late" is acceptable? And the issue is that "only" in that context doesn't mean a singular item, but rather a group of ambiguous size?