Having access to $HOME is already gone enough to produce irrecoverable damage, specially if there is Internet connection.
I trust Microsoft not to be literal malware, but I don't trust Microsoft to avoid doing stupid unnecessary crap to your Mac.
Moreover, it is also quite granular. You can allow access to "Downloads" but still have to grant permission for it to access "Documents" later.