I think a lot of us are trying really hard here to avoid the simplest and most obvious answer, which is that AI coding software is helpful, but doesn't deliver the kind of productivity gains boosters are talking about.

There's a bunch of reasons for this, but the obvious one to me is just that I haven't seen any backed up claims that actually break down how a 10x (or similar) speed up would get delivered in practice.

For the sake of an example, let's imagine an engineer who works an 8 hour day split up like this:

- 2 hours: investigating and debugging potential issues

- 1 hour: reviewing code

- 2 hours: in various team meetings, catch ups or otherwise

- 1 hour: designing (including at a level of writing code for prototyping etc)

- 2 hours: actually writing code

I think that's a semi-reasonable day breakdown. Realistically, I think AI helps debug faster, and generate code faster. Possibly design work too, but I think this isn't really the case since AI workflows necessitate a lot more design work in order to make the AI generation deterministic.

So looking at that day, if I was able to fully automate writing code and debugging issues so that it took 0 time at all (which I think is a very optimistic assessment of the actual speed ups AI provides), we're looking at a 2x speed up of an engineers productivity.

If we think AI makes those things twice as fast (so 2 hours, instead of 4) the engineer is now at a 1.25x speed up. That's great, but it's hardly revolutionary. It'll change your output something like the way going from a team of 4 to a team of 5 might do.

This is obviously complete conjecture and I'm sure the average time engineers spend on things is different, but I think there's an obvious take away. If you want to make claims about the performance impact of code, you need to know where the code spends its time, and what impact you're making on those sections of code.

Similarly, if you want to know what performance gains are being made, you need to know what engineers spend their time doing and what impact AI is having of those tasks.

I've seen very little attempt to actually make realistic claims on that basis, and a lot of vibes-based "wooah, AI makes me so much faster" type claims. Those claims are obviously not nothing, but I wouldn't trust people's gut reactions to make serious claims about the overall productivity changes in our entire industry.