Not only that but it makes sharing the website a chore. Half the people you share it to will complain that it's not working at all... and try telling someone "here's a random link, make sure to disable your anti-trackers to visit it!"
If a site doesn't work properly with GTM blocked, legitimate use case or not, personally I normally just move on. Unless I have a compelling reason to need that site, I won't make any attempt to work around it or change my blocker settings.
The only Google property I'm aware of that isn't routinely blocked by these kinds of tools now is Google Fonts, and I think some of them even block that by default now.