That is not an issue with the safety of the Vector, it is an issue with the safety of 'int' and raw pointers. If the Vector grows, that pointer points to freed memory.
But yes, in my implementation I have a safe version of int called 'i32' which overrides the & operator and doesn't allow it to return raw pointers.