A lot of issues like this end up being tied to one or two specific routers; in a previous issue, it was found to be ram with a stuck bit in a particular line card connecting ATT and a major CDN (or somebody) in Palo Alto [1]. Especially if it's an issue on a connection to CDNs, your VPN connection is likely to traverse a different path, and the path from the VPN to the destination will also likely traverse a different path.

Most of the time, networking issues are at a single link on the path, and forward and reverse paths are different too, so it can be hard to track down. BGP does a great job of quickly routing around totally broken links, but there's not usually any sort of quality or utilization metric included, so a link where there's lots of problems but BGP still works is going to stay up and continue to be utilized.

[1] See discussion here, but I didn't see the specific resolution https://news.ycombinator.com/item?id=25335936

confirming that using tailscale and an exit node in the google cloud seems to fix the problem for now