64 squares * (Empty | (Black | White) * (King | Queen | Bishop | Knight | Rook | Pawn)) + (Black | White) * (Can castle queenside | Can castle kingside)
It could be encoded as 64 4-bit int columns + 4 bool columns = 260 bits, most of which don't change between moves. Normal columnar encoding and compression strategies could probably reduce that to a few bits per board state.
The thing I optimized for is that there’s very often repeated blank spaces or repeated pawns on the same rank.
Also instead of storing castling status separately, it’s stored as a separate piece on the appropriate rook.
These take advantage of the fact that there’s 6 pieces, so 3 bits to encode them leaves two options remaining. One is PawnRepeating and the other is RookCastleAvailable, in my scheme.
There’s probably improvements to be made. I’ll write a post on it when it’s finalized.
I guess you also need some way to encode which player's turn it is. Though maybe you could eliminate that by flipping the board on black's move and always encoding from the perspective of the last player's move?
I'm curious about whether a naive columnar encoding scheme could beat a more complex encoding scheme, after compression. Not columnar in the sense of ranks and files, but columnar in the sense of data science storage formats (e.g. parquet), where each 'column' is the state of a specific square across different board states. Given 64 such columns, a game state is a single row. The hypothesis being that if you're encoding all the game states of a large number of games (say 1000), after RLE and compression etc you would see a net compression better than more complex encoding schemes. Given a big block of game states like this, then a single 'game' would be a list of offsets into the game states. This would probably also compress very nicely in columnar format.
Now I want to try it...