Hacker News Reader

Hacker News

Comment by shakna | Hacker News Reader

Parent Full thread

shakna

Once the header is read, you can iterate over the ReadableStream, though, can't you?

View on HN

1. You know the size of the compressed data from the Content-Length header.

2. You can iterate through the uncompressed response bytes with a ReadableStream.

Please explain how would you produce a progress percentage from these?

Recompress the ReadableStream to work out roughly how long the compressed version is, and use the ratio of the length of your recompressed stream to the Content-Length to work out an approximate progress percentage.

Lol! Upvoted for creative thinking.

Many large files that get downloaded are already compressed, so using compression during http response is pointless and just adds overhead.

If you need progress on a text file, then don't compress it while downloading. Text files that are small won't really need progress or compression.

If you're sending a large amount of data that can be compressed, then zip it before sending it, download-with-progress (without http compression), and then unzip the file in the browser and do what you need with the contents.

I'm sure there are probably other ways to handle it too.

>I'm sure there are probably other ways to handle it too.

Yes, there is. The simplest solution is called XmlHttpRequest, which has a "progress" event that correctly reports the response body bytes.

You can even make a wrapper for XmlHttpRequest and call it "myFetch()" if you insist.

That doesn't change the fact that it is possible to use fetch with progress.

Sure, it is possible to use fetch() with progress, as long as Content-Encoding is not used. Which is precisely what I've said in my first comment.

And I've explained why, in most cases, Content-Encoding doesn't need to be used for async download with progress monitoring.

If you had control of both ends you could embed a header in the uncompressed data with the number of uncompressed bytes.

"processed uncompressed bytes"/"all uncompressed bytes" is a distorted progress indication anyway.

Or put that length in the response headers.

You won't be able to do this if you're downloading from a CDN. Which is exactly where you would host large files, for which progress reporting really matters.

Why not? Every CDN I've ever worked with preserves custom headers.

Right. For example S3 supports custom headers, as long as that header happens to start with "x-amz-meta-..." - and now your progress reporting is tied to your CDN choice!

Not sure about you, but to me "XmlHttpRequest" in my request handling code feels less dirty than "x-amz-meta-". But to each their own I guess.

S3 is not a CDN.

Check CloudFlare, Akamai, ...

Good idea. We could give it a new MIME type too. E.g. application/octet-stream-with-length-prefix-due-to-idiotic-fetch-api

Or we can keep using XmlHttpRequest.

Tough choice.

Reply on news.ycombinator.com