1,009 karma · joined July 2, 2022
I suspect that this might be more of a "Mustang body kit" on a Tesla chassis and not retrofitting the Tesla tech into a Mustang chassis + body. Still cool, but maybe misleading.
It also helped build an intuition of what wach model could do and which parts it was weaker at because you could try them almost side by side, especially if one model's output wasn't great.
That said, these were all side projects so nothing truly consequential. Otoh, you might leave some extra perf on the table but I found the models worked quite with the Copilot harness.
That would mean earnings of ~ $1.1T would be required on that investment annually, so maybe on $2T of revenue, capturing 2% of the global GDP - so I'd estimate that GDP would need to go up more like 5-10% to justify this.
[1] https://substackcdn.com/image/fetch/$s_!Gf2t!,f_auto,q_auto:...
when i signed up for billing via cloud console and entered my credit card, i got $300 "free credits".
i haven't thrown a difficult problem at gemini 3 pro it yet, but i'm sure i got to see it in some of the A/B tests in aistudio for a while. i could not tell which model was clearly better, one was always more succinct and i liked its "style" but they usually offered about the same solution.
Even with our exorbitant energy prices, I pay about 1/3 for the cost of energy per mile (off-peak). I have to go in for service 3 times less often (and just that is worth a pot of gold to me). I no longer need to detour to gas stations once or twice a week as I can just plug in the car at night. And then EVs are also more enjoyable to drive imo.
I think this post comes in light of Europe being seen as curtailing free speech recently. Europe and the US have always had different ideas on the limits of free speech.
I ended shoehorned into backend dev in Ruby/Py/Java and don't find it improves my day to day a lot.
Specifically in C, it can bang out complicated but mostly common data-structures without fault where I would surely do one-off errors. I guess since I do C for hobby I tend to solve more interesting and complicated problems like generating a whole array of dynamic C-dispatchers from a UI-library spec in JSON that allows parsing and rendering a UI specified in YAML. Gemini pro even spat out a YAML-dialect parser after a few attempts/fixes.
Maybe it's a function of familiarity and problems you end using the AI for.
What I'm seeing in the suburban example graph in the article, is that the vehicle and hood have gotten way taller... I don't know how hoods/grills this high improve safety - I assume it's mostly the opposite. But they do "look rugged/beefy" - like all trucks and SUVs have to in order to sell - just look at the difference! [1]
"Millions of SUVs, trucks have dangerous front blind zone" [2]
[1] https://static1.hotcarsimages.com/wordpress/wp-content/uploa...
[2] https://www.nbcnews.com/news/us-news/americas-cars-trucks-ar... (or all the other writeups of this report)
I don't know how this program would be structured, but imo this program is not doomed to fail due to underfinancing - of course this being an EU program it surely has other issues.
Not sure if OpenAI has updated O3, but it looks like "pure" o3 (high) has a score of 79.6% in the linked table, "o3 (high) + gpt-4.1" combo has a the highest score of 82.7%.
The previous Gemini 2.5 Pro Preview 05-06 (yea, not current 06-05!) was at 76.9%.
That looks like a pretty nice bump!
But either way, these Aider benchmarks seem to be most useful/trustworthy benchmarks currently and really the only ones I'm paying attention to.
Right now (for me) it's very frequent, depending on the type of project, but in the future it could be less frequent - but at some you've gotta test what you're rolling out. I guess you can use another AI to do that but I don't know...
Anyway, my current workflow is:
1. write detailed specs/prompt,
2. let agent loose,
3. pull down and test... usually something goes wrong.
3.1 converse with and ask agent to fix,
3.2 let agent loose again,
3.3 test again... if something goes wrong again:
3.3.1 ...
Sometimes the Agent gets lost in the fixes but now have a better idea what can go wrong and you can start over with a better initial prompt.
I haven't had a lot of success with pre-discussing (planning, PRDing) implementations, as in it worked, but not much better than directly trying to prompt what I want and takes a lot longer. But I'm not usually doing "normal" stuff as this is purely fun/exploratory side-project stuff and my asks are usually complicated but not complex if that makes sense.
I guess development is always a lot of testing, but this feels different. I click around but don't gain a lot of insight. It feels more shallow. I can write a new prompt and explain what's different but I haven't furthered my understanding much.
Also, not knowing the codebase, you might need a couple attempts at phrasing your ask just the right way. I probably had to ask my agent 5+ times, trying to explain in different ways how translate phone IMU yaw/pitch/roll into translations of the screen projection. Sometimes it's surprisingly hard to explain what you want to happen when you don't know the how it's implemented.
Also, access to market leading AI is not going to cost $20/mo when everything is said and done.
[1] https://blogs.worldbank.org/en/developmenttalk/half-global-p...
This project is not your typical Webdev project, so maybe that's an interesting case-study. It takes a C-API spec in JSON, loads and processes it in Python and generates a C-library that turns a UI marked up YAML/JSON into C-Api calls to render that UI. [1]
The result is pretty hacky code (by my design, can't/won't use FFI) that's 90% written by Gemini 2.5 Pro Pre/Exp but it mostly worked. It's around 7k lines of Python that generate a 30-40k loc C-library from a JSON LVGL-API-spec to render an LVGL UI from YAML/JSON markup.
I probably spent 2-3 weeks on this, I might have been able to do something similar in maybe 2x the time but this is about 20% of the mental overhead/exhaustion it would have taken me otherwise. Otoh, I would have had a much better understanding of the tradeoffs and maybe a slightly cleaner architecture if I would have to write it. But there's also a chance I would have gotten lost in some of the complexity and never finished (esp since it's a side-project that probably no-one else will ever see).
What worked well:
* It mostly works(!). Unlike previous attempts with Gemini 1.5 where I had to spend about as much or more time fixing than it'd have taken me to write the code. Even adding complicated features after the fact usually works pretty well with minor fixing on my end.
* Lowers mental "load" - you don't have to think so much about how to tackle features, refactors, ...
Other stuff:
* I really did not like Cursor or Windsurf - I half-use VSCode for embedded hobby projects but I don't want to then have another "thing" on top of that. Aider works, but it would probably require some more work to get used to the automatic features. I really need to get used to the tooling, not an insignificant time investment. It doesn't vibe with how I work, yet.
* You can generate a *significant* amount of code in a short time. It doesn't feel like it's "your" code though, it's like joining a startup - a mountain of code, someone else's architecture, their coding style, comment style, ... and,
* there's this "fog of code", where you can sorta bumble around the codebase but don't really 100% understand it. I still have mid/low confidence in the changes I make by hand, even 1 week after the codebase has largely stabilized. Again, it's like getting familiar with someone else's code.
* Code quality is ok but not great (and partially my fault). Probably depends on how you got to the current code - ie how clean was your "path". But since it is easier to "evolve" the whole project (I changed directions once or twice when I sort of hit a wall) it's also easier to end up with a messy-ish codebase. Maybe the way to go is to first explore, then codify all the requirements and start afresh from a clean slate instead of trying to evolve the code-base. But that's also not an insignificant amount of work and also mental load (because now you really need to understand the whole codebase or trust that an LLM can sufficiently distill it).
* I got much better results with very precise prompts. Maybe I'm using it wrong, ie I usually (think I) know what I want and just instruct the LLM instead of having an exploratory chat but the more explicit I am, the more closely the output is to what I'd like to see. I've tried to discuss proposed changes a few times to generate a spec to implement in another session but it takes time and was not super successful. Another thing to practice.
* A bit of a later realization, but modular code and short, self-contained modules are really important though this might depend on your workflow.
To summarize:
* It works.
* It lowers initial mental burden.
* But to get really good results, you still have to put a lot of effort into it.
* At least right now, it seems you will still eventually have to put in the mental effort at some point, normally it's "front-loaded" where you have to do the design and think about it hard, whereas the AI does all the initial work but it becomes harder to cope with the codebase once you reach a certain complexity. Eventually you will have to understand it though even if just to instruct the LLM to make the exact changes you want.
It also looks like the second survey was sent out in June 2024 - so the data is 10 months old at this point, another reason why this it might be early.
That said, the latest round of models are the first I've started using more extensively.
The paper does address the fact that Denmark is not the US, but supposedly not that different:
"First, Danish workers have been at the forefront of Generative AI adoption, with take-up rates comparable to those in the United States (Bick, Blandin and Deming, 2025; Humlum and Vestergaard, 2025; RISJ, 2024).
Second, Denmark’s labor market is highly flexible, with low hiring and firing costs and decentralized wage bargaining—similar to that of the U.S.—which allows firms and workers to adjust hours and earnings in response to technological change (Botero et al., 2004; Dahl, Le Maire and Munch, 2013). In particular, most workers in our sample engage in annual negotiations with their employers, providing regular opportunities to adjust earnings and hours in response to AI chatbot adoption during the study period."
I would bet AI-native companies acquire their own cruft over time.
(Not to say that it takes openai years to train a new model, just that the timeline between major GPT releases seems to double... be it for data gathering, training, taking breaks between training generations, ... - either way, model training seems to get harder not easier).
GPT Model | Release Date | Months Passed Between Former Model
GPT-1 | 11.06.2018
GPT-2 | 14.02.2019 | 8.16
GPT-3 | 28.05.2020 | 15.43
GPT-4 | 14.03.2023 | 33.55
[1]https://www.lesswrong.com/posts/BWMKzBunEhMGfpEgo/when-will-...
Maybe
1. he's just doing his job and hyping OpenAI's competitive advantages (afair most of the competition didn't have decent COT models in Feb), or
2. something changed and they're releasing models now that they didn't intend to release 2 months ago (maybe because a model they did intend to release is not ready and won't be for a while), or
3. COT is not really as advantageous as it was deemed to be 2+ months ago and/or computationally too expensive.