These are one-time use tokens, just like you should use for a "Forgot Your Password" email login. It's literally the same steps, just there's no new password to input, the website just gives you a login cookie and you can go about your day.
If you'd want to use two-factor authentication by verifying access to the mailbox, you have to make sure that that (the connection to) the mail box is safe as well, which can potentially be very hard.
The mailbox still is a safe, but only from the moment that the mail has been sent to the moment the page has been requested. Mind that a safe connection to your mailbox does not also mean a safe connection to the server, but that is something that is covered by https+certificates (mostly).
TLS is already an under-spoken requisite of SMTP communications from any webserver doing "Forgot Your Password" workflows. TLS between mail servers is also already a de facto SMTP requirement for most bare minimum spam fighting mechanisms.
Let's not forget that most of the email system is still not encrypted (last time I checked anyway) even though the last mile between your mail server and your client may be.
There are also already a growing number of users that use "Forgot Your Password"-only login (most accidentally; human memory is porous) day to day.
Which is not to say that we shouldn't make email better and more secure, but that the security of our email infrastructure is a red herring of sorts in the larger security discussions of "passwordless" login, given that we are already equally insecure in every website that has a "Forgot Your Password" button that accepts anyone's email address.