> If using huffman, we must collect most strings and compute a static huffman tree.
Isn't that essentially what you've done to define your bespoke alphabets / partial encodings?
Isn't that essentially what you've done to define your bespoke alphabets / partial encodings?
5 bits 0 0000-1111 acde fghi lmno rstu
7 bits 10 00000-11111 bjkp qvwx yzAB CDEF GHIK LMNO PRST UVWY
7 bits 110 0000-1111 JQXZ 0123 4567 89/.
8 bits 111 00000-11110 !@#$ $%^& *()_ {}[] `~+= |\"' ;:<> ,? (space at the end)
16 bits 11111111 00000000-11111111 plus UTF-8 1 to 256 unicode characters encoded as UTF-8
There's some code here that could maybe be used for that sort of static huffman tree:
https://github.com/ryancdotorg/lztp/blob/main/generate-seed....
Alternatively, have something like this:
00 000000-111111 1-64 characters, 5 bits each
01 000000-111111 1-64 characters, 6 bits each
10 000000-111111 1-64 characters, ~6.4 bits each (character set of 84 characters, every 5 packed into 32 bits)
11 000000-111111 1-64 characters, UTF-8
As pointed out elsewhere, this will not outperform zstd with a dictionary trained on a corpus, but zstd would require pulling in a lot of code.
Doesn't QR use variable static encodings (alphabets)? e.g. mode 1 is numerics, mode 2 is uppercase alphanumeric, mode 3 is binary, and mode 4 is jis (japanese).