paytonjjones

This project was inspired by me having too much Codex budget and not enough project.

I set out to do two things: 1. Make a meaningful contribution to a game I like 2. 100% vibe a project (not look at the code, ever). I'd never actually done that before.

Given (2), I definitely wanted to make my own thing rather than pollute the PRs of any open-source maintainer.

After exploring some options I settled on making a really fast engine for Splendor. If you're not familiar with Splendor, it's a resource acquisition board game where you collect gems and spend them to make it easier to buy even more gems. There were a few Splendor bot projects out there, but not really any dedicated fast base engine for playing the game. Having a fast engine is broadly useful for playing, simulating, analyzing, and training.

Then, having built the engine more quickly than anticipated, I figured I might as well use the remaining budget to train a bot.

You can play against the bot here: https://splendorust.pages.dev/

I'm not fully sure if the bot is superhuman or not, but it does decisively beat the best open-weights Splendor [bot](https://github.com/lyquentxy/splendor) I could find. I suspect that [this one](https://github.com/inhabae/AhinLendor) is stronger than my bot but it's not open-weights. There's no dedicated bot vs. bot arena online — maybe that could be a fun future project.

Yes, memory loss surrounding the hours/days of the treatment is the major side effect.

Care to share the meta-analysis? How does 1/39 compare to the base rate of those with severe depression who did not receive ECT?

That does sound rough. The initial shock, without sedation, could cause mild to moderate pain and would certainly be a scary experience.

Even then, though, it's a far cry from being the antiquated mental lobotomy that many people wrongly assume it to be.

Do you think a potential bad actor would change their behavior if they took, say, a 12 hour course instead?

I am of course biased, but I have often been exasperated by the various "trainings", IRB processes, ethics classes, etc, that are required to run even basic psychology studies such as surveys.

Meanwhile, down the hall, the medical department is opening the skulls of mice, implanting electrodes, giving them supercancer, and then euthanizing them en masse. Often with less approval burden.

The hard truth is that it's tricky to decide what's ethical in science; we mostly allow or disallow things because they've been done that way for years, and all the added "process" is nothing but paper pushing and ass covering to quiet concerned outsiders.

I don't know the details here but I am a psychologist.

The "electric shock" normally used in psych studies is tiny, it's like the equivalent of one of those little toy prank buzzers. You couldn't possibly hurt someone with it, this is really a non-issue.

Related side note: Electroconvulsive therapy uses much larger electric shocks but, administered safely by a clinic, is extremely safe, has relatively minor side effects compared to e.g. SSRIs, and is very effective for severe depression. It's a huge shame that it's undeserved reputation prevents it from being more popular. Blame "One Flew Over the Cuckoo's Nest" for its inaccurate portrayal.

No, it's still separate. I don't know what that exploit was though, so possibly they just patched that.

> I believe ChatGPT Work sessions are billed against your Codex allowance, while ChatGPT Chat Sessions get their own, separate allowance. This may help explain the model availability differences

That's correct, and also why ChatGPT Work is DOA for me personally.

I need 100% of my Codex budget for Codex.

Too bad, it would have been nice to have a few extra features for the chat interface.

It's a good point, but in practice it depends on how easy those customizations are to implement / maintain, and how much money and effort you save. At some point the compute cost can disrupt even the nicest scale story.

I think the path forward is that websites offer one path for humans, and another for scrapers. But the huge catch is the path for scrapers must be _genuinely_ and _reliably_ the more economical and scalable path (either through something like PoW arms races, or through fear of litigation). Otherwise they will continue to ignore instructions and intrude on the human path.

In this case, because there's a vastly more efficient economic path for the adversaries (cloning).

They're not trying to engage in an arms race, they're trying to channel a racing river into its natural course.

I've had the same observation that Luna will quickly fill up its context window with reasoning, but it surprisingly hasn't been a problem really.

It will cycle through like 3 /compacts, complete the complex goal successfully, and cost me like 1% of my weekly usage on the $20 plan.

Edit: this is me using Luna directly, not Sol as the taskmaster

Your point is well taken, but one thing stood out to me.

> A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems.

This is actually my #1 problem with frontier LLMs. They are plenty smart, but they have no true long term memory, and their context window is not nearly enough for a complex system like a corporation. When my project gets big enough and I'm using a lot of /compact, the LLMs always have the same failure point: they fail to pay attention to something.

When they figure this out, I think it's going to be a very, very big moment.

The closer comparison would be 5.6-Luna. On DeepSWE at Xhigh it's 57% at 1/6 the cost of Sol M, on Max it's 67% at 1/3rd the cost.

Still an advance, I just thought it worthy to note Sol isn't nearly as impressive on the cost/performance frontier as discounted Luna.

It's a nice idea and I can see it working in some cases.

I'll be the first to admit I don't know the specifics, but I imagine in some cases, it's economically prohibitive to lower the price by such an amount that it makes up for the uncertainty in a typical search algorithm.

No, but I'd say it's slightly more likely than by chance (after all, success begets capital).

I actually prefer Dr. Pepper, but a lot of people think Coke makes the best cola, Apple makes the best laptops, etc.

> advertise everywhere you want

Except, apparently, the surface that's actually closest to the purchasing decision? Why must that surface be forbidden?

Amazon is a two-sided marketplace. Why should their obligation in search be 100% to one side?

Even for the buyer, there are second order consequences if you remove the ability for sellers to effectively sell. If sellers can't effectively ramp, they're going to turn to other platforms. If new sellers can't compete on Amazon, prices will increase and diversity of options will go down.

I'm not saying ads in search are a hard necessity in every platform. I'm just asking people to think through the whole issue before making snap judgments because ads are annoying to them.

> They know the best-reviewed, least-returned, best-priced model. The only purpose of the ads is to get you to pick an air fryer that isn’t that one

Thoughtless take. Say you've just invented a better looking, better functioning, cheaper air fryer. How do you expect to reach your first customers if not through ads? Sell it to your friends? That's even more invasive and annoying.

People's urge to rid the world of ads is similar to our urge to rid the world of "middlemen".

It's natural, and I get it (ads are annoying), but this is a forum where I expect people to think a little more deeply about the economics and second order effects of things.

Dystopia just decomposes to "bad place", of which there are many and varied types

Yes, institutional control features heavily in The Hunger Games, but so do "reckless uprisings, a long dark age, the collapse of institutions..." which is why it came to mind.

"Happy families are all alike; every unhappy family is unhappy in its own way"

There are many forms of dystopia and picking two that prominently feature absolute institutional control also seems like cherry picking.

The Hunger Games, World War Z, and Fallout are also dystopias.

It's honestly weird Claude converges on this language because it's incredibly wordy and hard to parse.

One would think semantic density would win out in training.

That's very interesting, I wonder if this applies also to models quantized to ints like (-1,0,1), and I wonder if the labs could maintain frontier performance if they removed floating points but arbitrarily scaled up the parameters.

Edit: the Thinking Machines article in the other comment gets into this a bit

For me, that unfamiliarity is precisely the point of reading something ancient.

Maybe it's just me, but "modern" and "context-aware" are the two things I most want to avoid when seeking out a translation of an ancient text.

There's a difference in the latency distribution between when you submit a query and you see the response, which is what the comment is (clumsily) asking about.

But yeah, there are a lot of factors, so it's hard to answer, and tokens/s isn't the right question.

Speedruns are for playing the track solo, right?

If you're playing normally (with other people and items), even the most skilled player can't avoid getting hit by some universal items and will benefit from acceleration.

Hence, as the author notes, the balanced build with Peach being favored among competitive players.

> there's nothing wrong with traumatic harm or abuse because people are resilient and it doesn't matter to their well-being

I know it's just a throwaway example, but this statement isn't internally inconsistent. Traumatic harm hurts people now, it doesn't need to permanently harm them to be considered immoral.

A very similar knee-jerk often occurs when people learn that parenting has very, very little effect on long-term psychological outcomes for children compared to genetics. But "parenting doesn't matter" is illogical for the same reason. "We may not hold their tomorrows in our hands but we surely hold their todays" - Judith Rich Harris

> correlations between group-level variables at the level of nations or states

Ecological mismatches in trauma harms is what I study, so I'll take the opportunity to shamelessly plug some of my and others' research, maybe you'll find it interesting.

[0] https://psycnet.apa.org/record/2027-56641-002 (open at https://osf.io/preprints/psyarxiv/9dpqu) [1] https://www.cambridge.org/core/services/aop-cambridge-core/c... [2] https://psycnet.apa.org/manuscript/2022-20702-001.pdf [3] https://dash.harvard.edu/bitstreams/e3528534-da4d-4402-a8b5-...

As someone who does _very_ related research, I'd push back on your skepticism.

I recommend Gilbert's "Stumbling on Happiness". It very accessibly addresses almost every point you raised.

In particular, people have very bad intuitions about trauma and loss. Humans are shockingly adaptable and bounce back to full functioning and happiness almost all of the time. Provided you don't have a biased sample (such as people seeking care) this is equally apparent when talking to victims of trauma as when doing empirical research.

Alpha School is a decent counterpoint to this.

Of course, it's a highly selected group of students. I very much doubt it's scalable in the way one would hope.

Maybe worth checking the data first...

https://ourworldindata.org/happiness-and-life-satisfaction