> the models are going to be much better than us at that, too.
I wonder if this is true. The code produced by these models are not really getting any more elegant over time. On contrary, the models seem to be getting worse, often proposing really baroque architectures. You can use RL to optimize for correctness, optimizing the vibe seems much more difficult.
The agents here very likely used search. On top of that, they have boundless patience and can quickly process top K hits to find what they need. This is exactly the skill that is super useful for finding various niche sub-proofs that can help you build the final proof. A human mathematician is not going to digest 1000 papers from a different sub-field to find the needle they want, not knowing if it is actually there. AI can do it in few hours.
The Iraqi invasion if famous for how little nation building was planned really. The US troops went in, overthrew Saddam, and then they had not idea what to do next. Nobody was really thinking about how to rebuild after Saddam is gone.
The most salient motivation was (perceived) security. The US government was absolutely paranoid after 9/11 and they really did not want Saddam to continue ruling in Iraq.
My guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.
The problem is that empirically regime changes very rarely lead to stable liberal democracies. Yeah, a magical societal transformation according to exactly our standards sounds great to us. But that is not how this is going to end.
How much money are Europeans paying to US companies for AI services and how much are Americans paying to EU companies? I don't think it's a good idea to let billions of dollars go to other countries because you couldn't be bothered to build a competitive product.
This is a bottom feeder mentality. Europe has enough bright people and resources to truly compete in the AI race. There is something wrong when the only selling point is that it's local.
Mushroom pickers generally know only a handful of easy to distinguish types and only pick those. I would say that 99.9% of pickers will never get poisoned.
> I disagree with this. Decoders were absolutely dominant in 2020 for chat. GPT2 was considered too dangerous to release, and I remember scrambling to get on the GPT3 waitlist. It worked.
He's not talking about decoders, he's talking about auto-regression. Before ChatGPT, the dominant paradigm was fine-tuning BERT-like models.
> Before ChatGPT there really wasn’t much of a concept of pre-training and post-training.
Again, people spend years just post-training BERTs in various ways.
There is no theory why the gap shouldn't be closed. At the same time, people are trying to make them work for years by now and it's never good enough. But they are competing with incredibly optimized architectures.
To me, all these agent systems just look very stochastic. You have these agents that have some basic computer capabilities and they are producing semi-random actions that also affect the semi-random actions of other agents. It is funny to observe how this stochastic system works, but it does not seem very practical to me so far.
The recent OAI-HF hack seems very similar. You have bunch of random actors and eventually they by chance iterated to a series of actions that breached HF environment. I don't perceive this as a malignant artificial intelligence, I perceive this as dangerous stochastic system that can control buttons that can affect the outside world.
The transformation from Roman republic to Roman empire was mainly about how the top-level government is organized. It was not about the relations with vassals/periphery. The Roman republic was as imperialistic as empire in this regard. In fact, most of the Roman conquering was already done when the republic ended.
I don't know the details of their implementation, but in general, if the text is not watermarked, you will probably fail the test after a few tokens. If it is, you will have to run the entire text, or, you can just run it until you have some degree of confidence.
And when you are an investor, you must predict who is able to make a good game. What is the likelihood that a Myst-like will pay for itself. There is already a lot of Myst-likes on Steam, do players want more?
The minimalist approach is great until it isn't. You can cover 98% of use cases with a single button is a great motto, until you are in the other 2%. I often fail to find basic functionality in software today, because some smart product manager decided that nobody really needs it. For example, it is now often impossible to load all elements from a list and filter/sort it along basic properties.
With the volume of outputs in today's academia, this is simply not possible. There are conferences with tens of thousands of submitted papers, grants have hundreds of pages, etc.
The AI companies also have a lot of space to grow their income (more ads, price hikes, ...). It seems realistic for them to turn profitable. But the market expected much more from these companies.
The standard of living in Rome was not surpassed for more than a millennium. Large parts of the empire were at peace for centuries, allowing economy and trade to thrive. Many provinces that were peaceful and prosperous were ravaged by wars, ethnic conflicts, and plagues for many centuries after the fall.
I understand that motive. On the other hand, LLM smell makes the text untrustworthy. I have detected it as well, and I immediately started to wonder about whether I am reading a reasonable expert analysis or just an AI hallucination. I still don't know.
I recommend prompting the LLM to mostly fix glaring grammatical and stylistic mistakes, not to rewrite the entire thing into a LinkedIn post style text.
There are many ethnic Russians in Eastern Ukraine that really welcomed the Russian army. The problem is that the rest of the country is not that happy about this.