It's beyond insane that Astra achieves above 50%. Are models allowed to "play against" the specific bot they're up against? What explains the fact that it drops to single-digits against strong bots?
9,124 karma · joined September 22, 2020
spunky.greener-0n@icloud.com
The point is that, especially for games perceived as being lower-status like MTG and other board games, I'm more inclined to believe the answer is closer to "nobody is willing to pour in the resources to seriously try" as opposed to "we definitively cannot with current science and technology."
There is nothing that, in principle, makes MTG different from poker or bridge, and we have superhuman engines for both.
One man's modus ponens is another's modus tollens I guess.
The model is just out. It could be good, great even, I don't know. But I do know that this index has Opus 5, one of the worst releases of 26, ahead of Astra. What information are we supposed to deduce from number having gone up?
The obvious ones: in Western sets the king has a cross on top, and Islamic traditions only use abstract shapes. But I wonder if the "usual" designs aren't at least somewhat influenced by the English language specifically. Bishops have a stylized miter and Rooks are a tower with crenellations, but e.g. in Russian Bishop is "elephant" and Rook is "boat". Even in many other Western European languages "Bishop" as in the piece is not the same as bishop as in the religious office, e.g. "fou" in French, "alfil" in Spanish, "Laufer" in German.
You need both sides though :)
What makes it interesting for estimating algorithmic complexity is that \log{n!} \in \Theta(n \log n). One side is obvious as you note, the other less so, but there's a famous trick to do both at once:
\log{n!} = \log{\prod_{h=0}^{n} h} = \sum_{h=0}^{n} \log{h}
Therefore,
\int_0^n \log{x} dx \le \log{n!} \le \int_0^n \log{x+1} dx
with both integrals trivial by parts.
>>>>>>
The new procedure targets inappropriate behavior by parents and spectators directed at children, coaching staff members, referees, match officials, or other parents and spectators. This includes, but is not limited to:
Verbal abuse (insults or profanity), as well as physical assault or shoving of one’s own children;
Verbal abuse, physical assault, or shoving of staff members of one’s own team;
Verbal abuse, physical assault, or shoving of players or staff members of the opposing team;
Continuous challenging, verbal abuse, physical assault, or shoving of referees and match officials;
Verbal or physical altercations with other parents or spectators.
<<<<<<
If that's the situation in, and it can't be stressed enough, a children soccer game, then yes, abandoning the match is probably the right choice. I assume they are only defining a formal procedure for practices that already exist.
For a superintelligent AI trained with methods anywhere close to what we are doing now, killing all humans is the default outcome. I actually think researchers are downplaying the risks to make their arguments more normie-friendly.
I'm also sticking to first-party agents, but that's more because I feel no compelling reason to switch and CLI agents fit well with my spartan technology choices. YMMV.
True.
> Regardless, the result is still valid (...)
If you think the game is rigged, the virtuous thing to do is to point that out and refuse to partecipate; making up your own rules is something I just don't understand, especially since the rule-abiding result would still have been SOTA.
> in the sense of passing the most famous benchmark designed specifically to measure AGI progress
The benchmark does not measure AGI progress or progress towards superhuman intelligence, as explicitly stated by the creators.
On the AGI question: surely you realize this depends on how we define the term? For example, one of the definitions OpenAI originally gave is "capable of doing most economically valuable work", which almost certainly Astra, as impressive as it is, would fall short of. I'm not saying it's a good definition, but as far as I'm concerned it's as good as any. More importantly, I don't think that it would change much if we said yes or no. I'm only bothering to take a position if it amounts to something.
This can feel as "moving the goalposts", and to some extent it is, but if done honestly "moving the goalposts" is how you make progress. Had you asked me 10 years ago I would have said that anything that could hold a conversation like GPT-4 could would probably have been wildly superhuman at almost everything. It shouldn't be hard to find ways GPT-4 was lacking, though. We see new things, we reassess and try again: that's how it's supposed to work.
To an extent I get it, this is not like a normal SaaS where you just buy seats, but their commercial offerings, especially B2C ones, feel like complete amateur hour. If they were selling literally anything else I'd have ran for the hills a long time ago.
Those problems are much worse in languages other than English, by the way. I'm not a native speaker but I always interact with LLMs in English because their prose in Italian is terrible, it often reads as though the text had been written in English and then translated.
Obviously there's nothing wrong with each of those stylistic elements in abstract, exactly like there's nothing wrong with em-dashes or the word "absolutely", but I would argue that writing well means choosing styles that match the content; what LLMs appear to be doing instead is forcing the message into a default pattern, which we percieve as forced and therefore tend to dislike.
To be clear: their writing is fine. The primary goal of writing is to convey meaning, and LLMs have been perfectly adequate at that pretty much from day one. I can get over their quirks.
AI probably changes the equation to some extent, but I still believe I'd rather maintain a complicated data pipeline like that in Python rather than R.
What polemic? Defining the determinant as the unique multilinear alternating form satisfying certain properties is very normal (and in fact the only way that really makes sense for both finite- and infinite-dimensional vector spaces). There are zero unusual things with this book imo.
Huh?
Mathematics is an extremely wide subject, it's perfectly normal even for two professional mathematicians not to understand each other's work. Have you considered that maybe they just don't work in that area?
Are you implying OpenAI's paper (which was, by the way, edited by humans and provided Lean certificates for most of the proofs) is actually gibberish? That's flat-earth levels of conspiracy.
The problem is that Paul Erdos was eccentric but he was also Paul Erdos. I'm not saying you're implying that, but I feel it's a similar line of thinking to how popular culture often romanticizes autism and Asperger's because some very smart people are (allegedly) affected. The same group includes people who need 24/7 care.
Computer science at the frontier is as specialized and hard as mathematics, you need years of study to truly understand a field well enough to make meaningful contributions. You can try sponsoring me if you really want, but I don't think you'd be spending your money wisely in expectation.
It's a very O-ring problem.
I tend to like simpler/textual interfaces though, admittedly this won't be everyone's cup of tea.
I mean yeah, that's mostly what I'm saying?
The only parts where I would disagree are that capabilities are spiky, and also the fact that they're a bit prone to losing the forest for the trees without guidance (the macro/micro distinction). But yeah, by default I don't write code by hand.