chr15m

You can simply look this up to verify it.

> and MTG

No. The current frontier models would not beat the best humans at MTG, even with Python scripts and web search. Unlike Chess, there are no existing engines that beat humans.

Yes it will feel very different.

Yes, capabilities are correlated but an open question is how correlated, and how much negative transfer (where the model regresses on other benchmarks) will come into play.

Why are frontier models not chess geniuses (amateur club level elos), despite knowing the rules deeply, having a great deal of chess data in the training set, and having general reasoning training in other domains? Similarly, a beginner can beat a frontier model at Magic The Gathering. Why?

It is because chess and MTG are not priority targets in the (expensive) training pipeline. Chess is already solved by lesser neural networks so there is no wow factor, and nobody cares if they suck at MTG.

Mathematics on the other hand, has a huge wow factor, and a large and highly verifiable dataset.

Related: why did OpenAI give up on video?

I agree it's disorienting and scary but I'd phrase it like this: "the moment we come up with a systematic, objective way to verify a task, *and we can generate or find large datasets, and somebody has a reason to pay for the expensive training*, we become obsolete at it."

The economics side of it is an important constraint, currently ignored because the big labs are swimming in money (debt).

Every dot in these graphs is composed of multiple benchmarks smooshed together.

Benchmarks are by definition a curated set of things that can be measured and evaluated.

Things that can be measured and evaluated can be (expensively) trained on.

The progress we are observing comes from figuring out how to measure and evaluate particular aspects of intelligence, and then expensively training on feedback loops made of those exact measurements, and synthesized data representing further measurements (literally benchmaxing).

What is not shown in these graphs are the set of things that intelligence can do, which are difficult, expensive or impossible to measure and evaluate.

That set of things is much larger than the set of benchmarks represented in the graph.

Therefore the post is misleading in the sense that the "superintelligence" being targeted is "jagged" to use Karpathy's term, and we infact already have jagged superintelligence (calculators, compilers, Bayesian spam classifiers, etc.)

We are witnessing a profound explosion of much more jagged superintelligence, restricted to the exact domains that can be (expensively) RL'ed on.

Mathematics is machine verifiable, meaning you can automate RL at high volume.

Because "miss the forest for the trees" does not have a large and easily machine-verifiable dataset.

Cool graphs. What does the top one show?

Anthropic's missing last quarter results in a nutshell.

> Why would the volume that any of this be increasing exponentially?

Surely it's obvious?

Slop cannons firing at a rate so high no human can hope to restrain them.

Such a colossal volume of slop that even the most well engineered, efficient technology on the planet, written by one of the greatest coders of all time, is no longer sufficient to contain it.

I understand https://neal.fun/infinite-craft/ uses an LLM under the hood each time a novel combination happens.

Godspeed, sir.

It's probably trained in a video game-like environment.

A sultry saxophone riff would free me from the human condition & constant struggles. Come on Genie!

Heck yes, Minecraft is very good for this! Highly hackable. I'm so grateful it exists so we can get more people excited in computers and computation and hacking and building.

Minecraft worlds also have low Kolmogorov complexity and high logical depth, producing these incredible complex worlds from a seed and a relatively small program. Just that observation is drilled into every player's experience in a visceral way and I think it's a useful thing for more people to know and appreciate. A kind of intuition for complexity and compressibility.

You raise a great point. The kids are alright!

Will check out your threads.

[Like you I also worry about short form video, but I note that in the past humans have got bored of these types of addictions so you might find we see it waning at some point.]

Such a good show. My kids grew up on it!

Wow cool!

Happy days. I think Prince of Persia is one of the only games I beat.

Did you ever play Another World by Erich Chahi? Still one of the greatest games and still good today.

I am lucky that weather is not a problem here, and yes we are grid connected. So I am 100% net positive on solar inference with 5kW.

Conan the Barbarian and Below the Root on our Apple IIe. Tiny worlds behind the glowing Orange CRT. How do I make something like this? My mum (lol) helped me figure out how to get keyboard input in BASIC. I made a little game where an O character chases your X character around the screen. Core game loop. Browsing pirate CGA games at the warung in Pondok Indah, well outside the jurisdiction of US law. Glorious rack upon rack of tiny worlds to explore. Death Track and Prince of Persia. More BASIC and that gorilla throwing bananas. Later on the Multiline BBS. Talking late into the night with tech weirdos twice my age. All manner of mind bending ideas. Newsgroups. Punks and skaters and cypherpunks. Predicting what was coming, seeing the future we expected unfolding. Downloading Commander Keen over FTP. WinSock and Netscape. My first homepage. Downloading Linux and getting it running. Configuring XFree86 modelines all night lol. Turbo Pascal and VAX VMS at uni. My first job coding. Hacking out web backends in gnarly ass Perl. No databases just blatting and slurping comma delimited text files lol. Getting paid to do the most exciting thing there was to do, all day. Magic.

You only need about 1kW to run a local inference machine. Even a 5kW array is overkill if you are using it during sunlight hours.

This could be significantly cheaper if they used a different model.

Yes, thanks!

People who are already using LLMs to drive Clojure REPLs say it works well (see #ai-driven-development on the Clojure Slack). I don't think forgetting which cells have been run is a problem with the described set up.

Interesting that you choose "regressive" as the opposite of progressive. thinking emoji

Damn good.

Yes but we're all biased differently. It's a rich palette.

Although LLMs are not deterministic in normal operation (because of the temperature values used) they are still strongly biased in their outputs. Bias in their outputs is literally how they function.

So you get a different type of conformism where people who interact with LLMs a lot are going to be influenced towards the median, as encoded in those token probability look up tables.

The effect is basically the same as what Heidegger and Illich were worried about, and I think we're seeing it happen already everywhere.

A solution(?) is to try to be your true weird authentic self when you act in the world, rather than bending your words and behaviours to the machine and LLM probability shape.

Gish gallop. Harder to argue with.

Software developers love complexity. They love to construct huge beautiful towers of the stuff.

Also it's the code version of this old phrase: ("I Had More Time, I Would Have Written a Shorter Letter") https://quoteinvestigator.com/2012/04/28/shorter-letter/

Changing banks is incredibly annoying but this might be a reason to consider it.

I have also faced this issue with banks and their frustrating authentication issues. Not to mention proprietary app requirements.