Caching across sites is a privacy risk in itself, because scripts can measure the time required to load a resource and therefore detect if a visitor has visited another site with the same resource before. That‘s why modern browsers no longer cache across sites.