* You could have gotten a lousy route due to poor traffic management or a network issue by your ISP, the transit providers, or a CDN
* You could have been given an overloaded server or POP by a CDN's DNS
* Your ISP could have a netflix or third party caching appliance, and it could be overloaded
* Your ISP could be under-backhauled, under-peered, or under-transited in any points along your route
* Your ISP's transit provider(s) could be under-peered, under-backhauled, or under-transited if they are not a tier-1 network. See also poor traffic management and network issues.
* Any party along the way to the CDN and to the origin could be under-peered, under-backhauled, or under-transited. See also poor traffic management and network issues.
Using a VPN proves very little. Various BGP and DNS tricks are used in the bullets above, and a VPN can drastically change the parameters of all of them.