For example, if IV1 and IV2 match and the block pattern is ABCDABCDABCD, then BACDBACDBACD will produce the same hash value.
A minor finalizer change would fix it for any IV (pseudocode as I don't actually know Rust) -
vec[0] ^= diffuse(vec[1]); vec[1] ^= diffuse(vec[0]); vec[2] ^= diffuse(vec[3]); vec[3] ^= diffuse(vec[2]);
u64 result = diffuse(vec[1] ^ diffuse(vec[3]));
that's probably overkill but should work.