That's why I put the word "potential" in there. Every allocation potentially needs to go to the OS for more pages.
And yet, they can't all not be in cache. As soon as the cache is accessed for one instruction, a bunch more are pulled in too.
Any given vector.push call could reallocate the vector, but it's amortized O(1).