Hacker News Reader

Comment by rockmeamedee | Hacker News Reader

Yes but most of it is NNUE, a custom neural network trained on mountains of data to copy modern Stockfish.

So a computer from back then _can_ run at Elo +3k, but the programmers at the time wouldn't have been able to implement it even if you went back with a time machine and the blueprint for the technique. They wouldn't have the teacher model to distill from.

View on HN

“In principle, I could have used the same engine itself as the teacher, but just to save on wall clock time I used Stockfish. This reduced the wall-clock time from ~4000 1996-core-years (something entirely doable in 1996---but sadly not something I can do with my single lonely desktop) to just a few days on a modern desktop running a modern engine.”

I see no reason to doubt the author here.

Kasparov wasn't allowed to train for 4000 years. /s

Actually, even pre-NNUE Stockfish wouldn't be an entirely fair comparison, because all hand-coded pre-NNUE heuristics was tested using Fishtest [0][1] - a distributed testing framework with compute, donated by volunteers.

And I think that Fishtest is actually what made Stockfish the strongest, because no other engine's performance ever was evaluated on such scale before.

Stockfish was the first to use hard data instead of vibes, when selecting algorithms and heuristics.

So, even if we add a rule that NNUE is not allowed, because it was trained on modern GPUs that, obviously, didn't exist back in 1996, hand-coded heuristics would also take a ton of compute to develop.

Bitter lesson strikes again, even when you don't expect it.

[0] https://en.wikipedia.org/wiki/Stockfish_(chess)#Fishtest

[1] https://github.com/official-stockfish/fishtest/wiki

> Stockfish was the first to use hard data instead of vibes, when selecting algorithms and heuristics.

I am half remembering a post from more than a decade ago but a process similar to Fishtest was supposedly used for Rybka. But in general I agree.

Yes, but the implication is about 'hardware overhangs'. It is colossally expensive in compute to develop the full history of Stockfish and NNUE and train the final SOTA checkpoints... But once you do, you can run it on even the tiny dumb chips like 'every smartphone' to get human-level chess performance on billions of nodes overnight. (Application to other domains of more real-world relevance than 'playing chess' left as an exercise for the reader.)

NNUE was added to Stockfish in 2020. Although this was a big improvement, it was already extremely strong before that. With plain hold hand-coded logic as a board evaluation function.

Stash has never adopted NNUE and breaks 3k with ease

It's also great reference engine with all of the pinned tags of its evolution.

https://github.com/mhouppin/stash-bot

https://chessprogramming.org/Automated_Tuning has excellent history. Still substantial compute on tuning, but viable.

You could have used one or more Deep Blue computers as the teacher model, or just a cluster of regular PCs doing very deep searches.

Doesn't the author address this point? They speculate that it could be attained through self play back then using a ton of 1996 compute and time, but it's theoretically possible.

Reply on news.ycombinator.com