different fonts give sites different personalities, adding richness to the web (whether you appreciate it or not). for 99.9% of sites, being overly focused on font size/download speed is an unnecessarily trivial optimiztion.
with that said, don't remotely load google fonts because privacy does matter. for most sites, the speed and bandwidth difference doesn't matter. just serve them as first-party, cacheable assets.