There was a game called Driving in Tehran which was exactly that. If you speed or crash, you get fined. If you hit someone, it tells you "don't play games with people's lives" and then exits entirely.
408 karma · joined November 27, 2022
There was a game called Driving in Tehran which was exactly that. If you speed or crash, you get fined. If you hit someone, it tells you "don't play games with people's lives" and then exits entirely.
I think part of why we underestimate difficulty is, it's hard to see anything but the happy path. The complications are harder to anticipate. And for each of the solutions to a problem, we also only think of the happy path to solving it. And so on.
It's an old quote that, although not 100% accurate anymore, still sums up my feelings quite nicely.
The Vanderbilts and Carnegies of today could do a major public service if they wanted to. Use their names and hire the best researchers in their respective fields and pay them well.
The rules are: 1) scalars always broadcast, 2) if one vector has fewer dimensions, left pad it with 1s and 3) starting from the right, check dimension compatibility, where compatibility means the dimensions are equal or one of them is 1. Example: np.ones((2,3,1)) * np.ones((1,4)) = np.ones((2,3,4))
Once your dimensions are correct, it's a lot easier to reason your way through a problem, similar to how basic dimensional analysis in physics can verify your answer makes some sense.
(I would disable broadcasting if I could, since it has caused way too many silent bugs in my experience. JAX can, but I don't feel like learning another library to do this.)
Once I understood broadcasting, it was a lot easier to practice vectorizing basic algorithms.
The problem is that for most of the interesting problems out there, it isn't possible to see every possibility let alone memorize it.
On chess specifically, my takeaway is that the branching factor in chess never gets so high that a breadth-first approach is unworkable. The median branching factor (i.e. the number of legal moves) maxes out at around 40 but generally stays near 30. The most moves I have ever found in any position from a real game was 147, but at that point almost every move is checkmate anyways.
Creating superhuman go engines was a challenge for a long time because the branching factor is so much larger than chess.
Since MCTS is less thorough, it makes sense that a full search could find a weakness and exploit it. To me, the question is whether we can apply breadth-first approaches to larger games and situations, and I think the answer is clearly no. Unlike chess, the branching factor of real-world situations is orders of magnitude larger.
But also unlike chess, which is highly chaotic (small decisions matter a lot for future state), most small decisions don't matter. If you're flying from NYC to LA, it matters a lot if you drive or fly or walk. It mostly doesn't matter if you walk out the door starting with your left foot or your right. It mostly doesn't matter if you blink now or in two seconds.
A couple of issues. First, the model completely botches the "Trike" brotherhood to "Tike", which also drops the meaning of the symbol. It's a tricycle. The seat isn't a sidecar, it's behind the driver's. Some of the numbers for the racecars also seem to be hallucinated.
It also doesn't seem accurate to describe the rewritten captions as having a "richer" vocabulary as the authors do. It's certainly more verbose and captures a lot of details about the setting, but lacks information density for the subject of the photo. You can't make up for low specificity with more high-probability words. Sometimes you need a long-tail word.
For example, the Western Kingbird caption. It's not just a grey and yellow bird. If the model doesn't know what a Western Kingbird is, you're not going to be able to draw a Western Kingbird no matter how verbose your description is.
But the first problem is, the number of legal transformations is actually infinite. (Maybe I am wrong about this.) So it immediately becomes impossible to search the full tree of possibilities.
Ok, so maybe a breadth-first approach won't work. Maybe we can use something like Monte Carlo tree search with move (i.e. math operation) ordering. But unlike chess/go, we can't just use rollouts because the "game" never ends. You can always keep tacking on more operations.
Maybe with a constrained set of transformations and a really good move ordering function it would be possible. Maybe Lean is already doing this.
I guess there might be a disagreement of whether the problems in ARC are a representative sample of all of the possible abstract programs which could be synthesized, but then again most LLMs are also trained on human data.
If I can make one criticism/observation of the tests, it seems that most of them reason about perfect information in a game-theoretic sense. However, many if not most of the more challenging problems we encounter involve hidden information. Poker and negotiations are examples of problem solving in imperfect information scenarios. Smoothly navigating social situations also requires a related problem of working with hidden information.
One of the really interesting things we humans are able to do is to take the rules of a game and generate strategies. While we do have some algorithms which can "teach themselves" e.g. to play go or chess, those same self-play algorithms don't work on hidden information games. One of the really interesting capabilities of any generally-intelligent system would be synthesizing a general problem solver for those kinds of situations as well.
If I can make an analogy to chess engines, a type of AI so old that people don't call it AI anymore [2], if we had started out by applying transformers to available chess games, we would never be able to create an engine that could reliably beat a grandmaster. Predicting the next chess move based on some representation of the game and its relationship with previously-seen positions only gets you so far. Deep Mind tried [3] this.
One route forward, which doesn't require a fundamental breakthrough, would be synthetic data. To stick with chess, we could create huge synthetic datasets of a huge amount of chess games, and simply continue training as we were. However, this is very expensive, and can be tricky if the synthetic distribution isn't the same as the real one. And the end result is still going to require incredible computation power to use.
What an LLM really lacks is the ability to search. And search is really important [4]: without it you'll probably need orders of magnitude more data and orders of magnitude larger models. At a fundamental level, search means we don't just make a snap judgement about e.g. the likely next token, but, as Daniel Kahneman would say, "thinking slowly" about what to do/say next.
Assuming we will not have search is how you get projections like needing 100 GW power plants to supply dedicated training data centers. The human brain uses 12 watts. 100 GW is the energy requirements of all human brains on the planet. And yet, if we put all human brains on the planet together, our collective capabilities would be outclassed by a chess engine I can run on my phone.
[1] https://arxiv.org/pdf/2406.02061
[2] https://en.wikipedia.org/wiki/AI_effect
So part of the reason may be that developers tend to have access to hardware which isn't representative of a typical user. I probably could've written a more clever algorithm at the time, but why bother for a one-off?
For example, suppose you have a lot of chess PGNs you'd like to store [1]. If you compress each PGN separately, you won't get nearly as high of a compression ratio as if you stored them all in the same file and compressed that file. But storing games individually is more convenient. Here are the numbers:
Original PGN file: 780 bytes
Default zstd compression: 508 bytes (35% decrease)
With dictionary: 376 bytes (52% decrease)
I'm thinking about ways to compress at an even higher ratio. Chess PGNs are a great candidate for this task because they have a huge amount of redundant and repeated text.