keeeba

Thank you for the pelicans sir, how do you think they compare to other models in Sonnet’s pricing/capability range?

Opus 5.1 came out about a month ago, what gives?

The elves left a long time ago, it’s difficult to see much virtue in what remains.

Also interested

This is not an attempt at affected nonchalance, but I’ve simply never come across the idea that Dostoevsky is particularly difficult to read.

A somewhat gifted teenager will race through it, as will an average adult.

Not hired Suleyman? Build his own research lab?

Satya made moves early on with OpenAI that should be studied in business classes for all the right reasons.

He also made moves later on that will be studied for all the wrong reasons.

The gap between Anthropic and the other guys keeps growing

Big respect

Total humiliation for Hegseth, sure there will be a backlash

To somewhat state the obvious - the problem isn’t the amount of data, it’s the algorithms.

We need to discover the set of learning algorithms nature has, and determine whether they’re implementable in silicon

Is everything OpenAI do/release now a response to something Anthropic have recently released?

I remember the days when it was worth reading about their latest research/release. Halcyon days indeed.

“ This is one of my fundamental beliefs about the nature of consciousness. We are never able to interact with the physical world directly, we first perceive it and then interpret those perceptions. More often than not, our interpretation ignores and modifies those perceptions, so we really are just living in a world created by our own mental chatter.”

This is an orthodox position in modern philosophy, dating back to at least Locke, strengthened by Kant and Schopenhauer. It’s held up to scrutiny for the past ~400 years.

But really it’s there in Plato too, so 2300+ years. And maybe further back

I don’t have the experiments to prove this, but from my experience it’s highly variable between embedding models.

Larger, more capable embedding models are better able to separate the different uses of a given word in the embedding space, smaller models are not.

Doesn’t seem like this will be SOTA in things that really matter, hoping enough people jump to it that Opus has more lenient usage limits for a while

As a fairly extensive user of both Python and R, I net out similarly.

If I want to wrangle, explore, or visualise data I’ll always reach for R.

If I want to build ML/DL models or work with LLM’s I will usually reach for Python.

Often in the same document - nowadays this is very easy with Quarto.

Oh boy, if the benchmarks are this good and Opus feels like it usually does then this is insane.

I’ve always found Opus significantly better than the benchmarks suggested.

LFG

Please don’t actually use these 5,6,7-way Venn diagrams for anything practical, they’re virtually useless and communicate nothing.

I agree it is a profound question. My thesis is fairly boring.

For any given clustering task of interest, there is no single value of K.

Clustering & unsupervised machine learning is as much about creating meaning and structure as it is about discovering or revealing it.

Take the case of biological taxonomy, what K will best segment the animal kingdom?

There is no true value of K. If your answer is for a child, maybe it’ 7 corresponding to what we’re taught in school - mammals, birds, reptiles, amphibians, fish, and invertebrates.

If your answer is for a zoologist, obviously this won’t do.

Every clustering task of interest is like this. And I say of interest because clustering things like digits in the classic MNIST dataset is better posed as a classification problem - the categories are defined analytically.

“Skills are a simple concept with a correspondingly simple format.”

From the Anthropic Engineering blog.

I think Skills will be useful in helping regular AI users and non-technical people fall into better patterns.

Many power users of AI were already doing the things it encourages.

It came from nowhere to 1T tokens per week, seems… suspect.

What use-cases do you see for the 270M’s embeddings, and should we be sticking to token embeddings or can we meaningfully pool for sentence/document embeddings?

Do we need to fine-tune for the embeddings to be meaningful at the sentence/document level?

Anthropic say Opus is better, benchmarks & evals say Opus is better, Opus has more parameters and parameters determine how much a NN can learn.

Maybe Opus just is better

How have you tested your recall in the long and short term? And what were the results?

Just checking my notes here.

This is the same Sam Altman who abandoned OpenAI’s founding mission in favour of profit?

No it can’t be

I want to believe that it wasn’t announced at that time, with that name, purely to detract from Google I/O.

But it’s hard

Nice story - I’ve seen you on the leaderboard a few times. Good luck through the rest of Foundations III

Nice, also find small classifiers work best for things like this. Out of interest, how many, if any, of the 3million were labelled?

Did you end up labelling any/more, or distilling from a generative model?

Thanks for linking - I know this is pedantic but one might think OpenAI’s models could make their content free of basic errors quite easily?

“Conretely, let's define a routine to be a list of instructions in natural langauge (which we'll repreesnt with a system prompt), along with the tools necessary to complete them.”

I count 3 in one mini paragraph. Is GPT writing this and being asked to add errors, or is GPT not worth using for their own content?