Spending 10 months building the most human-like chess bot
mbuffett.com
mbuffett.com
From the position history in the previous post. Good point. Same with bishops "sniping". If a bishop is moved to b2, and remains there or 20 moves, it can be "forgotten" and opponent suddenly has a pin or something underprotected. But if I saw the exact same board, but the previous move was moving the bishop to b2, I would be aware of it. That kind of local thinking is of course something I should learn to not do, but at my level (1700-1800 FIDE) it's how humans play.
It seems strange that even with fairly abundant datasets such as Lichess, that it is still difficult to approximate human strength with a bot.
As a “low strength human”, it’s very easy to see how traditional chess engines fail to emulate humans.
As far as actual calculation look ahead goes, humans aren’t calculating to chess engine depth. How often is a human really considering every possible move to greater than a 3 move depth?
Humans are moving with instinct. We cull the decision tree based on our evaluation of how promising a branch may be, not its objective calculated strength. This is where the chess engines break down.
Humans rely on instinct, pattern recognition, and strategy. None of this translates to a chess engine unless you are a top level player with near perfect instincts and pattern recognition.
My hunch is that for approximately human strength, the lowest ELO human makes moves completely at random, where the highest ELO human with follow the brute forced top lines. The challenge is tweaking the decision tree to match human evaluation of a branch based on skill level, and a second conditional probability of selecting a branch based on the player’s ELO.
The output of the chess engine shouldn’t be deterministic, but probabilistic. History has shown when you take a probabilistic approach to a problem, normally magic falls out. I’m yet to see a probabilistic chess engine.
And all of this also changes with level. I'm speaking of correct plans, drawing a technically drawn ending, and so on. If a player is weaker then complex for them might simply mean when there's lots of pieces doing lots of stuff, making them more likely to hang material to a simple threat. And the aforementioned RB v R ending might be trivial for them ( on the R side) because their opponents are not going to be strong enough to create plans that can apply pressure and try to make progress. They're just going to move their extra piece around hoping you blunder something.
So getting back to engine strength - I think figuring out what's complex for a human is a very basic prerequisite, amongst several, to making an engine that plays human-like. And we're just not there yet, or seemingly even especially close.
I also coach kids and none of their moves are random in the mathematical sense. They dont play the best moves but the moves are understandable once you hear their reasoning, it's just their reasoning doesn't properly evaluate the immediate position, future shapes and transformations.
For example looking at my chess.com profile I find basically every hung piece, and I am excellent at taking advantage of pinned piece opportunities and almost never allow an imbalance due to pinned piece, but I have blind spot when it comes to clearance sacrifices both taking advantage of them and allowing them.
So I suspect a good way to approximate a human would be to bork the evaluation based on a profile and then pick from that narrowed set of candidate moves at random. Which is more or less my understanding of how the chess.com bots work
IIRC there have been attempts to introduce human-like behavior like with Shredder which was designed to introduce human-like mistakes.
This is the most interesting part for me in part because it seems arbitrary.
Back to the recent post: > shapes the trunk; never feeds the policy logits
I didn't really understand that part.
As at 900 level, me playing white, it goes Bf3 in the opening as if I have a knight there, but there isn't a knight there anymore. I just moved it to e5. There's 3 pieces protecting that square. Would never see that in a real game except if it's a crazy premove (it did not move quickly).
At 1800 level, I'm about to double attack their knight on h6, so it moves its king out of g8 to get a flight square. But then it doesn't follow through on its plan, inexplicably ignoring my threat and giving me a piece. I have no other immediate threats.
The bigger annoyance now is that there seems to be a bug with premoves where the move registers (or appears to do so) but your timer just runs down and nothing can be done.
I was able to beat the max elo (2800) bot in bullet chess pretty consistently because if you manage to not get checkmated in some number of moves it just always runs out of time. Still fun though as it's kind of like playing an older GM who is playing on their phone.
Fixed the premove thing too, I think.
I haven't run into the premove issue anymore.