1,379 karma · joined March 13, 2025
what kind of nonsensical tonfoiling is this lol. it doesn't require a genius to figure out the instructions by crawling through HTML.
in fact, almost every LLM has prompt injection capabilities and will instantly tell you that there were prompt injections.
do you really think an LLM today is susceptible to these obvious tricks?
You could've made the same argument at any time in history because all technologies do the same thing. Things will go on, mastery will remain.
Its something I've noticed in many folks. You'd say something like "well sure, did you try using a different model" and you will be met with screeching. Its almost as if they've shut off facts and made up their minds.
An example: a year back I wanted to show that AI doesn't hallucinate that much.
Here's like a 20 comment long conversation with a person who believed that ChatGPT hallucinates easily. It took them a long time to prove it - they used toy examples and it did't work because they'd never used the thinking version. Then they had to resort to esoteric examples in the benchmark which still didn't work.
AI will make things cheaper and more abundant - at the cost of a few people not enjoying their jobs. So what.
1. coarse grained processing
2. fine grained processing
Tokens are coarse grained processing. While I agree that during a token generation, that processing is fine grained.
But Looping is fine grained processing or reasoning and it is not limited to discrete tokens within passes. This is qualitatively different because a lot more "stuff" happens between tokens. You are right that it is similar to a larger model but then even COT is similar to using a larger model.
In those cases, the feedback loop is more direct and not necessarily relying on toy proxies like LOC or documents. My point is that it would be like grading a VP of product on how many lines their document has. That'd be stupid because VP's are seen as more mature and have longer term incentives. I don't think it should be any different for employees.
TL;DR employees will become small business owners-ish with AI.
This is just a hypothesis, but I'm willing to be proven wrong.
This article is just saying that since dotcom bubble happened, there’s a mysterious thing that will force any new technology to be a bubble. But they don’t talk about what this force is.
According to them, a bubble MUST happen. It is ordained - no - it is biblically promised.
None of the revenues, mechanisms and technological abilities matter. At least Ed Zitron feigns to do some quantitative analysis with his slop newsletter.
Or is it just a collection of AGENTS.md docs?
I'm not sure why you left that out?
The maths community can continue doing what they want and not be bothered..
But coming to final answer is not important and that’s what the author spends time arguing. If they are right, surely whatever slop OpenAI is producing is worthless to human understanding which is the main goal.
The math community has to change prestige markers to promote people who contribute to understanding and not just solving theorems and that’s on mathematicians to do.
If solving problems is the end goal then I don’t mind OpenAI doing it in which case furthering math is already solved by someone.
The only failure mode is if mathematicians can’t get funding for just “understanding”. But it shouldn’t be that hard to do if their claim is right that their main contribution to society has always been understanding.
I’d argue that getting funding purely for understanding may be even easier to justify because the costs go down dramatically per unit understanding.
The author spends a lot of time trying to suggest that LLMs are not the end all be all and humans are still needed for understanding etc. Fine. I understand.
If it is so the case that LLMs don't help that much, surely it doesn't matter if LLMs are privately controlled?
How come LLMs are important enough that private control matters but also not enough for actually advancing mathematics significantly on its own?
In other words, you are almost always better off with LLMs even if it is privately controlled. The mathematics community can just ignore LLMs for the base case or use ChatGPT if they want. In either case, it is strange to suggest that things will get worse for them. Its as if they don't have the coordination or agency to adapt.
“Anthropic bad because automation of labour”
“Claude bad because it hallucinated that one time”
“This is a marketing post to hype up ipo numbers”
when you see that OpenAI is not releasing a new model due to safety/alignment reasons, you now know why they aren't doing it. hint: its not for marketing or fear mongering.
here's the reality: words change meaning. they are fuzzy. and when the words take on new meaning while just requiring a simple extension (in this case a computer doing a remarkably similar thing), complaining is just futile and childish.
The industry is surprisingly slow to adapt and I'm writing two capabilities I'd like standardised in the industry.
I'd like standardisation of Agentic tests (I know, it sounds terrible) but these are tests that are broader and capture fuzzy intent of the system. Unit tests, e2e tests are focused and hard to maintain. An Agentic test (or a behaviour test) is where the agent understands the underlying behaviour intent and creates scenarios on the fly and executes them.
For example, I'd imagine that Claude Code goes through multiple Agentic tests where Claude itself tries to use the software using desktop control. This would capture a much broader range of bugs and its much easier to maintain because you only need the test to have a description of the intent and not all the hardcore plumbing required in e2e tests and unit tests (like mocks ughh).
I'd like a framework where I could simply have something like SCENARIOS.md and the framework runs the scenarios by spinning up the common environment required for the test. The industry is severely lacking in this kind of capability.
The next thing missing is Agentic deployment verification.
So image that you are deploying your service and you'd need to verify multiple metrics are in shape before you promote it to all servers. Traditionally this was done either by manual checks by the developer (tiresome and prone to errors) or just having alerts on a few key metrics (not extensive).
Concretely, I'd like the agent complete access to all my metrics and logs and have it make the judgement to either promote it or not promote it or notify the developer.
You'd argue this is not required, but in my experience, having this capability gives me a lot of sanity. There are greybeards who love to gatekeep their responsibilities and would argue something like "just have good unit tests!!11!!" but that's silly.
Why these are not becoming standard or even required practices is beyond me. I'm guessing that there's a cohort of developers whose mindset is risk aversion and they are not great at intuiting what an LLM is good at and what they are bad at. Unfortunately, a lot of arguments are just silly like "but hallucination", "but just have deterministic tests", "don't overcomplicate things".
I love COT and gives me a good idea about the thought process. Sure it doesn’t work always
Downvoted. but in time it will be clear whats likely the cause of this
“Two things can be true at once” is not an interesting response for this imo.