Maybe use ETag or a similar mechanism? This way we'll need to contact the server, but we save bandwidth that would be spent resending the resource, which is still faster than what we have.

Given something like this:

    <script src="https://code.jquery.com/....." integrity="sha384-R4/....." shared>

(where 'shared' invokes the caching mechanism)

The browser sends a request for it with If-None-Match: "sha384-R4/....." header set.

I think this solves 99% of the problem:

If the integrity tag doesn't match the ETag of the resource, the server interprets it as out-of-date cache and responds with content of that resource. If the integrity tag matches the ETag of the resource, it will respond with '304 Not Modified'.

And that's the remaining 1% of the attack surface: basically the attacker wins iff the site can be tricked into serving a resource with the same ETag as the hash of his payload. We don't need to worry about collisions: even if someone uses ETags that match the form of subresource integrity tags without intending to, the attacker would still need to generate a collision, which is just as hard as finding a collision with any other hash. But if there are servers out there that will serve files with externally-set ETags then they'd be exploitable.