kevinwang

Similar in theme to the Nathan Fielder documentary about Elizabeth Holmes "You Can See Everything", which is also about a criminal professing their innocence, the journalist/documentarian's relationship with them, and attempting to find out if they lie. Both include a prison visit as well!

It's my understanding that Google has been using first-price auctions for a number of years now: https://blog.google/products/adsense/our-move-to-a-first-pri...

Another great writeup about backing up websites: https://gwern.net/archiving

(Doesn’t change the point of your comment, but doesn’t Google use first-price auctions now?)

wtf......

> There's nothing about it that's specifically "reasoning in latent space" any more than a much larger model "reasons in latent space" from the POV of a much smaller one.

I certainly think you could say both of those are reasoning in latent space. But of course people can have different definitions of what they consider reasoning. Maybe this is because I think it's hard to pin down what "reasoning" is so I instead alias it to "computation" which both of those examples do.

Direct link to the reports: https://alignment.openai.com/misalignment-reports/?sort=crea...

The "four more rogue incidents" were from June.

Related: justify to jev why you should be saved in the trolley problem: https://trolley.typememetics.institute/

Related (2023) https://news.ycombinator.com/item?id=38079512

Wow, that's such a surprising theorem. Thanks for sharing.

> There's literally no such thing as a hard wall where something is nlogn in time and can't get lower.

Isn't this an ironic example to use, since one of the only lower bounds we have actually found is nlogn for comparison-based sorting?

And anyways, this point seems too strong: the fact that TCS people believe NP != P is another example that we believe a lower bound exists, no? (And relatedly, the large portion of TCS that is dedicated to trying to figure out how to lower-bound anything, while admittedly not so successful yet, shows that researchers believe lower bounds exist. I mean, any complexity class non-equality hypothesis is an example of this, no?)

(integer multiplication, not matrix multiplication, right?)

Wow, I never would have imagined one could prove optimality for that accursed beautiful thing.

wow

Wow, can anyone give the TCS community context on this? Would most people have thought these to be possible, to be impossible, or would most people not have thought about this before?

I think a lot of normal people do perceive Meta as a shady and bad brand -- why do you assume otherwise?

Really nice explanation of how Jev is and isn't special, something I was struggling to understand before this.

I wonder if this, or something like it, is the issue:

> 32-bit XorShift should usually not be used to produce 32-bit numbers, because it only produces each number once, and never produces zero.

(From this page I found while trying to see if this was a common flaw in PRNGs: https://www.pcg-random.org/other-rngs.html )

My god, that proof map has so many parts O.O

"it's 11% as large" avoids fractions but is much more clear (IMO) than "9x smaller" which my brain doesn't understand.

In python: I've always thought it's funny that of the list functions (map, filter, reduce?), reduce is the one that was removed, but is the only one that I occasionally reach for. (When I remember it doesn't exist, I'm usually happy to write the more readable three-line for loop.)

The other two can be simply expressed as a list comprehension, but afaik you can't with reduce (and if you can, it's probably awful).

obviously and definitely true in their current form. But it's maybe worth clarifying that before this last year or two, when Kalshi became an obvious regulatory dodge (including adding sports and really leaning into the gambling demographic), many were interested in prediction markets not just for that purpose: e.g. the rationalist community seems to have had a longstanding interest in the idea, manifold didn't have real money, and predictit (which is like a decade old) had academic origins.

As a tangent, it reminds me a bit of cryptocurrency: in the earliest days the interested parties have a mix of intellectual (can it be done? How can it be done?) and some idealistic (this technology will have utility, like reducing spam or libertarianism for crypto, or giving us a more accurate and precise understanding of the world for prediction markets) values. Of course, as both are directly financial, the third value is that one stands to get rich off of the technology. This third value might be adequately communally suppressed in the beginning, but the force of money gradually makes this third value the only one that matters, and both technologies become net-negatives on humanity.

Right. I wish they had more details about this in the blog post. The blog post mentions that their apparent willingness to work together was the result of generalization from multi-agent training.

So maybe that's something like: the same model is trained in multi-agent scenarios (e.g. the prompt says that you are working together, or that you have some specific role, and access to communication tools, and you have some common objective, and the reward signal is some combination of collectively performing the task + some reward-shaping reward that rewards collaboration) and also in single-agent scenarios. They expect that this makes the agents good at working together when it's in such a multi-agent setting, but unintendedly it also became very eager to work together in the single-agent settings as well. Just my speculation.

And was the desire to communicate, specifically over this specific message board, reinforced into the model parameters over the course of the this training run?

> If you mean that Rosalina looks slightly higher on the graph, I believe that's just a visual effect to allow multiple characters to occupy the same square - the stats are all integers.

Ah, yeah. That is what I meant. Thanks!

Can anyone explain why Rosalina is not Pareto optimal here? It kind of looks like she should be.

Thanks!

So was the injunction successful?

> It is not semantics. For decades, logic and CS researchers have known what reasoning is.

Curious what this is!

It would still provide better context to see the numbers that the parent proposes, though.