The fastest GIF does not exist | Hacker News Reader

The fastest GIF does not exist

biphelps.com

657 pointstodsacerdoti77 comments

An interesting quick dive into how some of the artifacts of historical implementations can carry forward decades later. I'm saving this one. While everyone on HN is probably familiar with a pile of examples of this in our own work particularly if you do anything lower level, this is a nice example because it's really easy to grasp even for total non-programmers and gives a feel for the layers upon layers the digital world is built on. Details on the x86-64 bootstrap sequence are interesting to tech people but this is the sort of thing I might show in a school class.

That said at first I thought the author was being a bit tongue in cheek on the suggestion to fix it with:

>A downside to making any changes is that the badly formatted GIFs won't render any more. I reckon just be bold, like the release of Netscape 2.0...If Netscape was fine with breaking poorly-made GIFs, we should be too!

But we kind of have a bit more content now then when Netscape 2 was released, and it seems they're serious?

  Support more than 256 colours in a single frame of GIF animation
  Support fast GIFs (10ms delay)
  No confusing behaviour with small delay = slow GIF
  Better compression for GIFs with multiple small areas updated per frame

But frankly at this point why? We've got good universally adapted image formats that do more than 256 colors. We've got webm and so on (even Apple caved on that one in the end) for baked animation which is much smaller and more powerful than gif, and plenty of CPU and libraries for generation on the fly. We've got powerful ways to do animation purely with HTML/CSS/SVG/JS. GIF is a historical artifact with a lot of its value at this point purely about old things. This wouldn't even retroactively bring newer capabilities to older systems because it would require updated code which by definition means the ability to run new browsers and thus modern capabilities.

I get they really enjoy hacking on gif but come on. Only way I can see it working is if all the players agreed on some metadata that could be added to gifs which wanted 'standards' vs 'historical' playback with the default remaining historical, and potentially also make it a browser setting. That'd still leave the ad issue, but maybe at this point that's moot given the advancements in blockers (and it'd be a very easy thing to look for as an indication of an ad, and thus be selected against).

> We've got webm and so on

The spiritual successor to animated GIFs — by that I mean nonlossy pixel-perfect animations — would be APNG, which is supported by virtually all current modern browsers: https://caniuse.com/apng

"nonlossy pixel-perfect animations "

That sounds way less awesome, when you consider it only had 256 colors, so allmost all content did indeed loose a lot, by converting to gif.

And this is actually the first time I have heard of apng and interesting that it is even now widely avaiable. But I do not really see a broad use case, compared to webm.

Because file size matters usually much more, than pixel perfect animation.

I wanted to convert a short screencast to a gif, result: hundreds of megabyte big monster, with crappy quality.

The same with webm: 2 MB and good enough quality.

The use-case isn’t video, it’s small icon-style color animations or animated pixel art (like the example in the article). It’s the same difference as between JPEG and PNG.

Well, yes - and for this use case I consider apng.

But my point was, that the main use case of gif, were short videos and animations, people shared. And for this, I am glad that webm seems now broadly avaiable for that role.

Well… that use case only started after the use of animated GIFs for small pixel-art animations already existed for many years. I always considered the video use, after it started, as a misuse/aberration. I certainly agree that webm is the right tool for the video use case. But I care about the original use cases of animated GIFs. If you want to get past the limitations of the format for those use cases, APNG should be the answer.

"If you want to get past the limitations of the format for those use cases, APNG should be the answer"

Sure, we agree on that.

> That sounds way less awesome, when you consider it only had 256 colors, so allmost all content did indeed loose a lot, by converting to gif.

Nonlossy can be used for pixel perfect animations, lossy formats cannot.

> Because file size matters usually much more, than pixel perfect animation.

It depends. For video, that is true. For user interface, any glitch is distracting. Using JPEG for user interface images like buttons is in most cases a bad idea. Each format is useful in different situations.

> I wanted to convert a short screencast to a gif, result: hundreds of megabyte big monster, with crappy quality.

You are right, that is not a use case for GIF or equivalent formats.

I suspect you could use JPEG as a lossless format, you'd just have to write your own custom encoder.

In some sense, JPEG is just a really weird programming language. So a JPEG file is just a really weird program that gets interpreted by a JPEG decoder. Most normal JPEG encoders try to find a short 'program' in the 'JPEG programming language' that gets interpreted by the decoder to recreate a particular image reasonably well.

But there's no reason you couldn't ask for a perfect reproduction.

As far as I know, JPEG decoders are lossless and deterministic.

I asked this question on https://stackoverflow.com/questions/71201691/does-jpeg-allow... to get something definite.

Of course, you point still stands that JPEG is not a good file format for storing UI elements. Even if in theory you could torture the JPEG standard enough to make this barely work.

"You are right, that is not a use case for GIF or equivalent formats. "

It is not, but this was still the main use case of gif - sharing short video sequences. With crappy quality, but it played everywhere. Apng is not the successor to that. Webm is, or will be.

So there might be niche cases in UI or games, where apng might make sense, but in most cases, I doubt people would see a difference between apng and webm - but they will notice faster or shorter loading times.

> when you consider it only had 256 colors

Per-frame palettes allow you to hack this a bit if you don't mind a few fractions of a second draw time for a static image¹² or only parts of the frame need to update each time for an animation.

Edit: some examples... A good illustration of the technique can be seen in https://commons.wikimedia.org/wiki/File:FullColour.gif and https://gif.ski/ is a practically pointless but brilliant tool for making high quality gif videos.

[1] or don't mind some dithering on the initial draw with the more colours added in subsequent “frames”.

[2] though for a static image, just use PNG

Sadly the support for APNG isn't great. For example on Discord GIFs works, but not APNGs. I also know of another service where, due to internal caching (IP hiding), APNGs get broken and don't animate.

It should work because discord app uses electron, so it must be on purpose or oversight from discord's part.

Discord's media display stack seems to be broken (for lack of a better word) in some very naïve ways. For example, embedding files with the ".gifv" extension (used on several popular websites to denote gif-like animations which are encoded in a modern video codec) doesn't work, even though they're valid video files expressing a webm or mp4 mime type. It instead displays a static image, which means they're reading in the video, then generating a thumbnail and ignoring the fact it's a video.

I heard that APNGs used to work, but then people just uploaded their APNGs instead of using emotes, and using emotes from different servers is the main selling point of Dicord's freemium "nitro" AFAIK...

Another reason could be that Discord caches images on its servers to hide viewer's IP from the original server hosting the image. The caching involves compression that sometimes breaks the image.

we have better formats but gifs are still everywhere, maybe there should be a gif222 to fix this stuff and maybe improve compression a bit too

That's kind of the point. What makes you think that people still using suboptimal GIFs are going to want to go update those GIFs to comply with breaking changes in how the spec is interpreted?

people won't change but if the gif library were updated then maybe browsers could support a modern rev with unambiguously defined behavior and compression improvements, maintain compatibility for older gif87a and gif89a files, and then a path would exist for gif authoring tools to eventually output gif222 by default at some point in the future (instead of throwing our hands up and letting this crusty old tech define our internet forever)

If only.

GIF is still the only format that supports animated frames, and is supported in most applications that support images. I sometimes create some animated WEBP to post somewhere, but the outcome is always the same, I have to go back and convert it to GIF or it won’t display.

https://en.wikipedia.org/wiki/APNG

I like the ”Internet Explorer 5 is closed source, but could hypothetically look something like this” bit.

Is this source code available through MS partnership programs or an old leak?

I don't believe the source code is still available officially (although it used to be through Microsoft's Shared Source Initiative and Government Security Program), but the Windows 2000 source code was leaked back in 2004.

https://news.microsoft.com/2004/02/12/statement-from-microso...

Windows 2000 and NT 4 code leaked in 2004.

I assume IE6 source code (mshtml.dll) was leaked along side with WinXP

I recall there was a leak a long time ago, but definitely well after the IE5-era. Hypothetically, it could be this leak where the code comes from.

It's in the Windows 2000 leak.

Its pretty trivial to disassemble binaries back into C++, but its against ToC so you wouldnt admit to doing so.

This has real comments, white space, and meaningful variable names. It’s vanishingly unlikely to be decompiled C/C++ code in this case.

Pretty easy to find the code on Sourcegraph: https://sourcegraph.com/search?q=context:global+fake+one+up+...

The repos I checked have all been DMCA’ed on GitHub, though.

DMCA'ed?

GitHub is owned by Microsoft, does it even make sense for Microsoft to DMCA itself? It can just remove the content, or leave it, it owns the copyright and the platform, it can do whatever it wants (probably not, but IANAL).

Or maybe it happened before the acquisition.

A couple of other gems in that file:

    // Okay, bug 30784 time. It's possible that we only got 1
    // measly byte in the last data block. Rare, but it does happen.
    // In that case, the additional byte may still not supply us with
    // enough bits for the next code, so, as Mars Needs Women, IE
    // Needs Data.
    if ( end >= _gifinfo.lastbit && !_gifinfo.get_done )

I was confused by this, I wondered on what basis they inferred that code. Never occured to me it might be leaked.

There was a source code leak in 2004: https://slashdot.org/story/43253

Rate limiting or clamping "0" delays upwards can also help for polling applications where you want to be as fast as possible, but you're never going to be actually zero. For example if you have a thread that's pulling images from a camera (publishing at 30 fps) in a loop, you can delay for 1ms between polls and nobody is going to know, but that can have an enormous impact on CPU usage; especially on embedded platforms. Instead of running at 100% thread usage (calling camera.read() and waiting for it to return true), the spin/loop cost is basically free and you only need to worry about the cost of acquiring/decoding the frame. In theory branch prediction should help because you'll take the "no image yet" branch 99% of the time, but in practice that just makes the loop iterate faster. I learned this the hard way writing a custom camera publisher in ROS, but I've seen it a lot in tutorial code and I think a lot of beginners make that mistake.

Author here - loving all the discussion! I was a bit intimidated to submit this to hn - I'm humbled that someone else thought it was worth posting. Of course, happy to answer any questions anyone may have!

> Interactive Bit

> You'll need to enable Javascript to see this section sorry! There's a slider that lets you see how changing the delay in a GIF file will affect the end results. It's pretty cool.

Thank you: this is an excellent way of handling the situation.

I'm glad someone came across that! Happy you thought it was a good way to handle it

It’s not very common for people to handle these sorts of situations elegantly. I disable JavaScript by default (mostly for performance, frankly), and I encounter such elegance only once or twice a year.

(For myself, I take care on my site in the few places I ever use JavaScript, and also even vary my content in feeds where it depends on JavaScript or site styles, which I’ve never noticed anyone else do. make-and-git-diff-test-harness and dark-theme-implementation are slugs of two sample articles that both vary by JavaScript and feed, I won’t drop links.)

Great writing and research. You added great examples to make it clear, both in code and gifs. It was both fun to read and interesting. I was surprised to see Netscape running, I might try as well to play with the first JavaScript.

Thank you! I used VirtualBox running Windows 95, but there were a lot of roadblocks - finding a Netscape 2.0 installer, mounting it etc.

Unfortunately I didn't document the process at all and have forgotten everything, so I can't give much useful info, apologies. Just know that there is pain down that path.

I thought it was very interesting. I'm also mad right now that we could have had much better looking gifs (and better optimized gifs) for so long!

The internet in all its duct tape glory.

Prime example of trade-off between cleanliness of implementation and backwards compatibility. As much as I like a pristine, pure spec, the market winners are clear. It pays to keep product debt around.

If we could only deprecate features more aggressively, our code would be so much cleaner...

flutter and Dart are essentially this applied to the DOM and JavaScript

And then you get a HN thread linking to some 10 year old bugzilla thread about how firefox doesn't support non UTF-8/ASCII encoded websites.

If anyone is interested in reading the GIF specification, I reformatted the plain text version into HTML: https://www.nayuki.io/page/gif89a-specification-html

Neat! Is there an automated tool for the conversion? Preferably hooked up to a RFC database?

I'm not aware of any tools; I converted this document by hand in a few hours. GIF is a popular standard with cultural value, which is why I would do this; I wouldn't do it for any random obscure document. The IETF RFC website has links to HTML and PDF versions, but...

For most standards documents, the HTML and PDF versions look just like the plain text version, adding little or no value (e.g. https://www.rfc-editor.org/rfc/rfc1321.txt ; https://www.rfc-editor.org/rfc/rfc1321.html ; https://www.rfc-editor.org/rfc/pdfrfc/rfc1321.txt.pdf ).

The only standard I am aware of with a decently formatted version is DEFLATE ( https://www.rfc-editor.org/rfc/rfc1951.txt ; https://www.rfc-editor.org/rfc/rfc1951.pdf ).

My favorite piece of GIF trivia is that in the original spec, GIF animations would play once and only once. Netscape used the Application Extension block to add number of loops or loop indefinitely. An Application Extension requires an application identifier. So basically every animated GIF ever made has the string `NETSCAPE2.0` in it. https://en.wikipedia.org/wiki/GIF#Animated_GIF

Thank you, this is wonderful!

IMO a better, though more complex, solution would be to keep the frame delays as specified in the GIF, but to disable GIF looping if the sum of all frame delays was less than 10 ms. That way you get the one good use case (>256 colour gifs) but malicious or broken GIFs with 0 frame delays don't hang your machine. This is complicated by the fact that browsers (used to?) display the first frame of a GIF before the complete animation was loaded. But you could handle that easily enough by making this decision at the time of first loop rather than at initial GIF load.

(Fortunately, this is all largely irrelevant nowadays.)

> We follow Firefox's behavior and use a duration of 100 ms for any frames that specify a duration of <= 10 ms.

That is interesting & funny. I wonder why there wasn’t just a minimum value.

This article doesn’t talk about what happens after the delay is triggered, or that a value of 10ms won’t work for the average browser, video hardware & monitor. That’d be a refresh rate of 100 frames per second, and a lot of monitors are 60Hz refresh. Don’t browsers default to 60Hz refresh as well? (MDN docs mention requestAnimationFrame callbacks are usually 60/sec, or 16.7ms apart, for example.) This means with a timeout of 20ms, you’ll probably get a janky animation where one frame gets doubled every 6 frames. I guess with GIF it’s not possible to solve that since the spec doesn’t allow you to match typical refresh rates.

Yeah, as a gamedev I was surprised not to see this in the article. There's often a default assumption in renderers that "60 fps" is the baseline (and most typical devices will be limited to that). Obviously gaming devices and newer smartphones push above that to 120, 144 or 240 fps. But in gamedev you're always faced with this problem of update frequency of anything that will be displayed to the player; if you update too fast (or too slow), you're gonna get "stutters" and skipped (or stuck) frames as the state updates faster than the renderer can render it (when too fast), or at a rate that's out of sync with the actual frames being pushed to the monitor (too slow, at a rate that's not evenly divisible by the frame rate).

I don't know why the animated gif seems as popular as ever, I can't think of any positives about it... bad colours, large file sizes and are encoders more readily available than proper video encoders? Also, supporting it fully now requires either threading or an event loop. The value isn't worth the complexity.

They usually work where jpg and png images work too, most importantly in most places where you render Markdown. No matter how more efficient a real video file would be, often a short gif right there in the README.md is the perfect way to convey a piece of information.

It's still a reasonable format for its original use - images with large areas of solid color where only a small percentage of pixels change across a small number of frames.

If your animation has <256 colors then it's even lossless, something not possible with any of the video codecs.

>something not possible with any of the video codecs

Which codecs? You can have eg. lossless H264 just fine.

Lossless h264 requires 4:4:4 chroma, which is allowed by the spec but not supported by many consumer playback methods. (QuickTime Player, for example, only supports the yuv420p pixel format.)

It is part of the High 4:4:4 Predictive profile, but you can use it with 4:2:0 subsampling if you want. It's more accurate that the all profiles H.264 defined kinda sucked except for Main, and no one really thought 10bit, 422 or especially 444 were useful and completely ignored those profiles until it was too late to be worth implementing.

Actually, lossless video is also rather marginally useful, especially given the decode cost of the arithmetic coding H.264/HEVC/VP9/AV1 have.

Ironically for your example, lossless, 4:4:4, and 10bit H.264 actually will play back on the new M1 Macs in QuickTime Player, though with a warning about it.

H.264, H.265, VP9, and AV1 all support losless encoding (regardless of color count).

Webp is the more literal replacement for the GIF use case though and also has no color limitations in the lossless animation mode.

Partly because of how they're presented to the user. You send a video, there's usually a play button, a bit of a delay to load, and sound. Gifs play automatically, first frame loads with barely any pause, and there's no sound ever. So, they present two different use cases, which, at the end of the day, have nothing to do with the format itself. But it can show that to humans, often the ends matter, not the means.

It's also a UI issue. On most OSs I can copy/paste an image like it was text. And a gif counts as an image so I can copy/paste images without having to save them. And since many modern UIs are hostile to saving videos, its the difference between holding and copying vs having to take a screen recording or finding a video download website to rip it.

It's really too bad that browsers did not add support for silent auto-playing looping videos to <img>.

Are they though? If I remember correctly, most "gif" sites online serve webp...

For the user the proposition is the opposite. The value of the technically superior formats isn't worth the increased use complexity.

Given that the reason for the forced delay is historical abuse by ads, I'm wondering if back in the days this didn't lead to an arms race between browsers and ads (I wouldn't know because I always use an ad-blocker). For example, after canvas was introduced ads could use those instead.

If you think that's crazy, there are other things like this that browser vendors do that are not standardized, and when you start writing a renderer to figure out why your output doesn't match, you find all of these silent agreements everywhere.

If you know, you know.

I converted the GIFs to APNG and both ffmpeg and ffplay gave the same results as the browser, so the specification is very consistently ignored.

So pretty much the same as this: https://webaim.org/blog/user-agent-string-history/

This "in practice spec" is a cool dive into history.

There is a similar quirk in the RTF spec where x and y coordinates were swapped.

What's the deal with the "hypothetical" IE5 source code that looks very much like actual IE5 source code?

There have been leaks over the years.

I think it's a joke, and the excerpt taken from a leak without admitting that.

Along the lines of 'downloading Linux distribution ISOs'.

I checked ImageMagick, and was somewhat surprised they don't note this at all.

On this page, for example: https://legacy.imagemagick.org/Usage/anim_basics/ , scroll down to "-delay", and it just says "Set the time delay (in 1/100th of a second)..."

I've got to say that the dancing dog doesn't have the same pizzaz if it's cropped like that, without his friend.

I really wish that meme would just die. That dog was sick and probably in a lot of pain and it makes me sad every time I see it.

There is no GIF faster than "no GIF".

So, of course it doesn't exist!

Reply on news.ycombinator.com