That's a very good point. At this rate of development I wouldn't be surprised if libjxl becomes x264 of image encoders.
On the other hand, libvpx has always been a mediocre encoder which I think might be the reason for disappointing performance (I mean in general, not just speed) of vp8/vp9 formats, which inevitably also affected performance of lossy WebP. Dark Shikari even did a comparison of still image performances of x264 vs vp8 [0].
[0] https://web.archive.org/web/20150419071902/http://x264dev.mu...