Modification time can be altered, or incorrect. File size is just another fast check that can be done with the same data return by stat'ing the file.

Rsync can also compare hashes if you don't trust the size and time on the files.