Hacker News Reader

Comment by amelius | Hacker News Reader

My preferred image format looks like:

    byte 0-3: magic code
    byte 3-10: 64-bit index into global decoder table
    byte 11-...: compressed data

The global decoder table is maintained by a standards organization. For every occupied index, it contains a piece of WebAssembly code that decodes the data and produces an image and metadata. Advantage: no more dealing with various image formats and installing decoders libs; there is just 1 format that can be tuned to specific uses and it will be available hundreds of years into the future. The WebAssembly code can be hand-optimized into native code (or even hardware) for frequently used decoders. The only downside is that you might occasionally need to access the central code repository over the internet to download a decoder.

View on HN

Cynically speaking, what you describe is more or less "take any file, prepend four magic bytes". The "index into global decoder table" is exactly what magic bytes are for in the first place. In other words, you're basically describing the thinnest possible container format.

Whether you provide the file format encoding/decoding as wasm or C reference implementation is not going to matter much in practice outside the web sphere, imo. You may argue that having a standard of wasm implementations is innovative, but if you're inside something browser-like, that's more or less the same as "just pull in something.js to decode this".

I don't think "download a new decoder from the internet occasionally" is as small of a problem as you make it out to be - in this case, an application could just as easily pull an update of itself to support a new file format directly. And you still get the essentially the same inertia as now for adoption of new formats because someone somewhere might not be able to update decoders.

Then there's also the issue of "how many copies of all those wasm blobs are you gonna have around on your system if every application uses this format". Which means you're now re-inventing system libraries in js/wasm. Talk about javascript eating the world...

But, optimistically speaking, your proposal could make speed of iteration in image file formats faster by streamlining adoption of new formats. Which could be a good thing.

> Cynically speaking, what you describe is more or less "take any file, prepend four magic bytes".

I was wondering if someone would notice that ;) What it amounts to is basically a package manager for image formats.

But it's the idea that matters here, not the implementation. If we want our data to be readable in 100 years from now, we better base it on some highly standard common format. That format could be WebAssembly.

I'd say its more or less an auto extractible image format, but wasm. Which would make sense!

Applied a bit more surgically, this is actually a pretty good idea. Take for example PNG: the header for the actual image data contains a one-byte field titled "compression method" with only one valid value: 0 for deflate. The idea at the time was probably that deflate is great, but something better might come along. Now almost three decades later we have plenty of better compression algorithms, but still all PNGs are deflate, because anyone who sets that header field to anything other than 0 breaks compatibility with every other PNG implementation that already exists.

If instead PNG was implemented as a definition of the metadata format, the "filter" stuff for making pixels more compressible, and a compression field that references a webassembly implementation of the compression method then libraries could still use their own optimized implementation for compression methods they know, but fall back to downloading the webassembly version for any method they don't know natively. You would have perfect backwards compatibility and could evolve the file format as technology advances.

The advantage of webassembly in that context is of course that it's trivial to sandbox, and the snippets don't need access to anything but their input.

>and metadata

You glossed over the most difficult part of the specification.

That screams "supply chain attack risk".

You find someone to pollute the "global decoder table" and you can introduce all sorts of fun problems, many of which turn into "I can't replicate it here (because I cached a working decoder, or the malicious decoder is only shipped to specific clients)"

TBH, I don't think we want external entity injection and Turing-complete languages in our image formats.

You also create a bunch of permanent assumptions in any of the encoders. Today, the "image data" it expects to return might be, say, an uncompressed 48-bit RGBA bitmap. What if the next generation of image formats can do more or different? Say, volumetric image formats for VR or holographic systems? You'd need a way to signal what you're getting in the metadata, which would then have to be open-ended enough to satisfy future needs, AND reliably enforced. (I'm imagining decoders that don't bother explicitly specifying all the return format properties, because in CURRENT_YEAR, only one type made sense).

> TBH, I don't think we want external entity injection and Turing-complete languages in our image formats.

Meanwhile, people are using WebAssembly and Javascript all the time, from untrusted sources.

If that is your taste, you should definitely take a look at zpaq [1] which uses an embedded bytecode with a not-too-long specification [2]. It is tuned for compression algorithms but any general algorithm can be programed; for example there is a project that turns zpaq into (slower) Brotli [3].

[1] http://mattmahoney.net/dc/zpaq.html

[2] http://mattmahoney.net/dc/zpaq206.pdf

[3] https://github.com/pothos/zpaqlpy

Including a futureproof decoder in every single image is a huge filesize overhead though. Or maybe I don't exactly understand your proposal.

The decoder-part is the same for many images, so the idea is to store them inside a standardized codebook that is pointed to from the data.

IIUC he is saying that there will be a 64-bit index into a global standards-maintained table, and the decoder is downloaded from said table. Not that the decoder is actually included with the image. The decoder itself would probably eventually be cached if not shipped with browsers by default.

Reply on news.ycombinator.com