I made a byte-range cache for object storage

I made a byte-range cache for object storage.

For full hits it's zero copy. On a miss, it fetches only the missing bytes from object storage needed to serve the request.

It merges overlapping ranges and coalesces concurrent misses into one object storage request.

I use it primarily for a full-text search implementation that uses tantivy which was adapted to be transactional and run on object storage.

With our implementation, a full-text search only requires 2 round trips to object storage. The first trip gets the manifests so we know which blob files we need to fetch for the given search. The second trip gets the blobs determined by the manifests in parallel.

Each request first hits the range cache so that rather than downloading the whole split file (which can get large 100s MBs), it either gets a hit and returns, or has a miss and gets the specific bytes it needs.

If some of the data from an object storage request overlaps the data already in the range cache it merges it so you have continuous byte arrays to read from. e.g. let cache = {some_key: [0..100]} and a new request for `some_key` retrieves [50..200] from object storage, the new data in the cache would become {some_key: [0..200]}

Thought it was pretty neat and that it could be useful to some folks who are working with object storage.