The hope with the TPM is that the system boots to a standard login screen, and the thief doesn't know any user's password. Much like someone snatching a laptop that's in 'suspend' mode.

Of course, a thief could try to bypass the login screen by e.g. booting with a different kernel command line, or a different initramfs. If you want to avoid this vulnerability, TPM unlock can be configured as a very fragile house of cards - the tiniest change and it falls down. The jargon for this is "binding to PCRs"