joshmlewis

Yes this is essentially my take I posted. We have been working at various levels of abstraction for a long time using framework, packages, and even programming languages themselves.

I'm surprised by a lot of the commenters here. People that are saying just wait for all these codebases to crumble for us to be needed again refuse to acknowledge the reality of what is happening around us. If that is your take, I advise you to become really, really good at programming and work somewhere in the realm of critical infrastructure or financial institutions where presumably agent written code will be last to permeate.

I do get the sentiment but the conclusion doesn't make sense to me. We have been solving problems as engineers at different layers of abstraction for a long time. Pre-AI, I used frameworks that abstracted lower level concepts away which made it faster to develop the solutions that mattered (aka solving problems for the end user). I could, given enough time, write the underlying database adapter to convert my business logic form into a database query and back out but I didn't need to because someone else abstracted that part away. Did I know how well that adapter was written? No. Were there bugs in it? Occasionally you run into a limitation or bug the author hadn't considered that required a patch or fix. Going a step deeper, the programming language itself is an abstraction that compiles into machine code or bytecode. Do I understand machine code or kernels? Definitely not. Some programmers do, especially if you have a compsci background, but I'd wager the number has been decreasing over the years. The point is, now agents are writing code and they are now pretty damn good at it. Better than the "average" programmer out there. Now we get to "program" in natural language by telling the agents what we want and they get to worry about nailing the syntax and all those details while we focus on the higher level decision and direction of what's happening. If your agent written code is sloppy and terrible, that's a skill issue not an agents write bad code issue. You need to setup the systems that result in code to your standards with the checks and balances to test and QA appropriately before it hits production.

Complaining about the state of things while sitting back with your arms crossed refusing to adapt to this bold new world is a risky strategy. People saying just don't use AI and do your job as normal are also going to have a difficult time. The devs that can leverage AI effectively and get a lot done will out leverage you to the point where it doesn't make economical sense to keep paying you to not use AI (unless you are in one of those super critical fields I mentioned earlier and even then I'm not sure those are safe). Isn't it more fun to embrace this new technology and figure out how to use it effectively vs complaining and refusing to acknowledge it? Help shape the future instead.

This is essentially what I've been comparing it to as well. You are laying out and being responsible for the overall direction, architecture, and how things should work at a high level but then delegating the implementation and lower importance decisions to the agents. It very much at times feels like managing a bunch of developers and coordinating them all toward the vision in my head. I've developed an intuition of sorts where I can quickly tell if something is being done incorrectly just by looking at the snippets agents show me and the questions they ask.

As engineers our job is to solve problems at various levels of abstraction. Developers that think we are just automating away our own jobs have missed the point. It's not about writing the code anymore, it's about designing solutions to problems and distilling that to agents in a way that achieves the outcome you wanted. Models are not good at coming up with this direction on their own, nor are they good at designing cohesive user experiences across a plane of complexity. These two areas require most of my time now while actually writing the code has been delegated away.

It's interesting they use output tokens as an eval because all tokens are not made equal. Even from model to model (like Opus 4.6 to Opus 4.7) the tokenizer can be different and it's no longer an apples to apples comparison. No one really talks about this but it directly affects stats like usage limits. Certainly comparing models between providers on an apples to apples comparison token wise is not a good test.

Speechify has been good for me although there might be better / cheaper alternatives I'm not aware of.

I think the OP was implying that it's probably already baked into its training data. No need to search the web for that.

"They" being the guy (Peter Steinberger) who created it as a personal project that he open sourced.

This is cool but as someone that's built an enterprise grade agentic loop in-house that's processing a billion plus tokens a month, there are so many little things you have to account for that greatly magnify complexity in real world agentic use cases. For loops are an easy way to get your foot in the door and is indeed at the heart of it all, but there are a multitude of a little things that compound complexity rather quickly. What happens when a user sends a message after the first one and the agent has already started the tool loop? Seems simple, right? If you are receiving inputs via webhooks (like from a Slack bot), then what do you do? It's not rocket science but it's also not trivial to do right. What about hooks (guardrails) and approvals? Should you halt execution mid-loop and wait or implement it as an async Task feature like Claude Code and the MCP spec? If you do it async then how do you wake the agent back up? Where is the original tool call stored and how is the output stored for retrieval/insertion? This and many other little things add up and compound on each other.

I should start a blog with my experience from all of this.

This just feels like the whole complicated TODO workflows and MCP servers that were the hot thing for awhile. I really don't believe this level of abstraction and detailed workflows are where things are headed.

This should not really be necessary and is more of a workaround for bad patterns / prompting in my opinion.

How big is your claude.md file? I see people complain about this but I have only seen it happen in projects with very long/complex or insufficient claude.md files. I put a lot of time into crafting that file by hand for each project because it's not something it will generate well on its own with /init.

It's also not a coincidence that Slack is neutering the ability to access channel history via the API very soon. With a very generous rate limit of 2 requests per minute I believe it was and a max of ~10 messages. This is already enforced for new marketplace apps and will apply to all apps starting in March according to their docs.

One of the biggest nuggets people need to take away from this:

> At one point we tried “improving” the prompt with Claude’s help. It ballooned to 1,500 words. The agent immediately got slower and dumber. We went back to 103 words and it was back on track.

Keep your prompts / agent instructions short. Focus on the wide view, not specifics.

As someone who builds AI products and having used agentic coding tools since they came out (often with Rails projects), I don't get this. There was a similar project called Rails MCP Server which said:

> "This Rails MCP Server implements the MCP specification to give AI models access to Rails projects for code analysis, exploration, and assistance."

And again I don't get the value. I can see some slight benefits to having a tight browser integration but I don't think that's worth leaving the IDE / CLI tools and the workflows they bring. You can also use Playwright MCP or just screenshot easily for more context. Claude Code can now run your server in the background and view logs as well. In a perfect world where LLM's can one shot whole features, maybe. But I can't let Claude Code go 10 minutes without it introducing a bad pattern and having to stop it. Reducing that visibility even further with this does not seem like a good combo.

I'm not wanting to tear down others projects either, just giving my perspective. I should try it to see how it does in the wild but the Copilot license or Anthropic API key requirement also deters me as well as having to have a project specific dependency.

It is funny how it can be like this sometimes. I think a lot depends on coding styles, languages, prompting, etc.

Cursor

When it came out on Tuesday I wanted to throw my laptop out of the window. I don't know what happened but results were total garbage earlier this week. It got better the past couple days but so far with gpt-5 being able to solve problems without as much correction I'm going to use it more.

Whoosh, it went right over my head.

The data is made up, the point is to see how models respond to the same input / scenario. You're able to create whatever tools you want and import real data or it'll generate fake tool responses for you based on the prompt and tool definition.

Disclaimer: I made PromptSlice for creating and comparing prompts, tools, and models.

I would highly doubt it. Even when you BYOK inside of Cursor they still say it's routed through their servers.

I noticed it was taking awhile on the first large-ish task I gave it. I'm assuming it was just a bit overloaded at the moment.

Where'd you get 720 from?

Did I say GPT-5? I said o3. :) That was a rebuttal to you saying you have never needed to add your key to use an OpenAI model before.

It seems to be trained to use tools effectively to gather context. In this example against 4.1 and o3 it used 6 in the first turn in a pretty cool way (fetching different categories that could be relevant). Token use increases with that kind of tool calling but the aggressive pricing should make that moot. You could probably get it to not be so tool happy with prompting as well.

https://promptslice.com/share/b-2ap_rfjeJgIQsG

It's free in Cursor for the next few days, you should go try it out if you haven't. I've been an agentic coding power user since the day it came out across several IDE's/CLI tools and Cursor + GPT-5 seems to be a great combo.

It does seem to be doing well compared to Opus 4.1 in my testing the last few hours. I've been on the Claude Code 200 plan for a few months and I've been really frustrated with it's output as of late. GPT-5 seems to be a step forward so far.

It does really well at using tool calls to gain as much context as it can to provide thoughtful answers. In this example it did 6! tool calls in the first response while 4.1 did 3 and o3 did one at a time.

https://promptslice.com/share/b-2ap_rfjeJgIQsG

I've been testing it against Opus 4.1 the last few hours and it has done better and solved problems Claude kept failing at. I would say it's definitely better, at least so far.

You had to use your own key for o3 at least.

> Note that BYOK is required for this model. Set up here: https://openrouter.ai/settings/integrations

https://openrouter.ai/api/v1/models

It's more efficient with tools for one and the input cost is cheaper (which is where a lot of the cost is).

See comparison between GPT-5, 4.1, and o3 tool calling here: https://promptslice.com/share/b-2ap_rfjeJgIQsG.