What’s missing is a way to tell the browser to use a certain resource (identified by its hash) and a url to download it.
That way the browser could cache the object across websites and download from a source that the requesting website controls.
It could look like this
hash://sha256/71dc8dd15ea156387fa9efaf380e7f8d39187a549649bec62a39aee3cf4fdc55//https://raw.githubusercontent.com/cdnjs/cdnjs/2c95eae0dc3569f0cb15c8a5f9173220b49156bc/README.md
I also vaguely remember that magnet links were similar.