The 100 hour gap between a vibecoded prototype and a working product
kanfa.macbudkowski.com
kanfa.macbudkowski.com
My thoughts on vibe coding vs production code:
- vibe coding can 100% get you to a PoC/MVP probably 10x faster than pre LLMs
- This is partly b/c it is good at things I'm not good at (e.g. front end design)
- But then I need to go in and double check performance, correctness, information flow, security etc
- The LLM makes this easier but the improvement drops to about 2-3x b/c there is a lot of back and forth + me reading the code to confirm etc (yes, another LLM could do some of this but then that needs to get setup correctly etc)
- The back and forth part can be faster if e.g. you have scripts/programs that deterministically check outputs
- Testing workloads that take hours to run still take hours to run with either a human or LLM testing them out (aka that is still the bottleneck)
So overall, this is why I think we're getting wildly different reports on how effective vibe coding is. If you've never built a data pipeline and a LLM can spin one up in a few minutes, you think it's magic. But if you've spent years debugging complicated trading or compliance data pipelines you realize that the LLM is saving you some time but not 10x time.
Examples: AI really wants to use Project Panama (FFM) and while that can be significantly faster than traditional OO approaches it is almost never the best. And I'm not taking about using deprecated Unsafe calls, I'm talking about using primative arrays being better for Vector/SIMD operations on large sets of data. NIO being better than FFM + mmap for file reading.
You can use AI to build something that is sometimes better than what someone without domain specific knowledge would develop but the gap between that and the industry expected solution is much more than 100 hours.
The only way to make LLMs useful for now is to restrain their hallucinations as much as possible with evals, and these evals need to be very clear about what are the goal you're optimizing for.
See karpathy's work on the autoresearch agent and how it carry experiments, it might be useful for what you're doing.
Man, I wish this was true. I know a bunch of non tech people who just trusts random shit that chatgpt made up.
I had an architect tell me "ask chatgpt" when I asked her the difference between two industrial standard measures :)
We had politicians share LLM crap, researchers doing papers with hallucinated citations..
It's not just tech people.
It took a lot of back-and-forths with her to convince her that the numbers she uses every day are "Arabic numerals". Even the author of the spec could barely convince her -- it took a meeting with the Arabic translators (several different ones) to finally do it. Think about that for a minute. People won't believe subject matter experts over an LLM.
We're cooked.
It's no different than conspiracy theorists. We saw a lot more with the rise in access to the internet. Not because they didn't put in work to find answers to their questions, but because they don't know how to properly evaluate things and because they think that if they're wrong then it's a (very) bad thing.
But the same thing happens with tons of topics, and it's way more socially acceptable. Look how everyone has strong opinions on topics like climate, rockets, nuclear, immigration, and all that. The problem isn't having opinions or thoughts, but the strength of them compared to the level of expertise. How many people think they're experts after a few YouTube videos or just reading the intro to the wiki page?
Your PM is no different. The only difference is the things they believed in, not the way they formed beliefs. But they still had strong feelings about something they didn't know much about. It became "their expert" vs "your expert" rather than "oh, thanks for letting me know". And that's the underlying problem. It's terrifying to see how common it is. But I think it also leads to a (partial) solution. At least a first step. But then again, domain experts typically have strong self doubt. It's a feature, not a bug, but I'm not sure how many people are willing to be comfortable with being uncomfortable
I've worked at places where ~5us was considered the fast path and tails were acceptable.
In my current role it's less than a microsecond packet in, packet out (excluding time to cross the bus to the NIC).
But arguably it's not true HFT today unless you're using FPGA or ASIC somewhere in your stack.
So yeah there's really no HFT anymore, it's just order execution, and some algo trades want more or less latency which merits varying levels of technical squeezing latency out of systems.
I don't work for a firm so don't get to play with FPGAs. I'm also not co-located in an exchange and using microwave towers for networking. I might never even have access to kernel networking bypass hardware (still hopeful about this one). Hardware optimization in my case will likely top out at CPU isolation for the hot path thread and a hosting provider in close proximity to the exchanges.
The real goal is a combination of eliminating as much slippage as possible, making some lower timeframe strategies possible and also having best class back testing performance for parameter grid searching and strategy discovery. I expect to sit between industry leading firms and typical retail systematic traders.
It wasn't a matter of choosing Java for HFT, it was a matter of selecting a project that was a good fit for Java and my personal knowledge. I was a Java instructor for Sun for over a decade, I authored a chunk of their Java curriculum. I wrote many of the concurrency questions in the certification exams. It's in my wheelhouse :)
My C and assembly is rusty at this point so I believe I can hit my performance goals with Java sooner than if I developed in more bare metal languages.
Then things like the jit, by default, doing run time profiling and adaptation.
You want low deterministic latency with sharp tails.
If all you care about is throughput then deep pipelines + lots of threads will get you there at the cost of latency.
how exactly you are passing data? You can pass some primitives without allocating them on heap. You can use some tiny subset of Java+standard library to write high performance code, but why would you do this instead of using Rust or C++?
Strangely this is one of the areas where I want to use project panama so I might re-implement some of the ring buffers constructs.
You allocate off heap memory and dump data into it. With modern Java classes like Arena, MemoryLayout, and VarHandle it's honestly a lot like C structs.
I answered "why" in another post in this thread.
my opinion is that no, it is not, declaring and using C struct is 20x times more transparent, cost efficient and predictable. And that's we talking about C raw stucts, which has lots of additional ergonomics/safety/expression improvements in both c++ and rust on top of it.
In terms of speed, memory usage, runtime characteristics... sure there are better options. But if java is good enough, or can be made good enough by writing the code correctly, why add another toolchain?
"writing code correctly" here means stripping 95% of lang capabilities, and writing in some other language which looks like C without structs (because they will be heap allocated with cross thread synchronization and GC overhead) and standard lib.
Its good enough for some tiny algo, but not good enough for anything serious.
those have low bar of performance, also they mostly became popular because of investments from Java hype, and rust didn't exist or had weak ecosystem at that time.
You have to optimize your memory usage patterns to fit in CPU cache as much as possible which is something typical Java develops don't consider. I have a background in assembly and C.
I'd say it's slightly harder since there is a little bit of abstraction but most of the time the JIT will produce code as good as C compilers. It's also an niche that often considers any application running on a general purpose CPU to be slow. If you want industry leading speed you start building custom FPGAs.
That is clearly false. I’m only familiar with Opus, but it quite regularly tells me that, and/or decides it needs to do research before answering.
If I instruct it to answer regardless, it generally turns out that it indeed didn’t know.
> Please carefully review (whatever it is) and list out the parts that have the most risk and uncertainty. Also, for each major claim or assumption can you list a few questions that come to mind? Rank those questions and ambiguities as: minor, moderate, or critical.
> Afterwards, review the (plan / design / document / implementation) again thoroughly under this new light and present your analysis as well as your confidence about each aspect.
There's a million variations on patterns like this. It can work surprisingly well.
You can also inject 1-2 key insights to guide the process. E.g. "I don't think X is completely correct because of A and B. We need to look into that and also see how it affects the rest of (whatever you are working on)."
"Ok let's look at these issues 1 at a time. Can you walk me through each one and help me think through how to address it"
And then it will usually give a few options for what to do for each one as well as a recommendation. The recommendation is often fairly decent, in which case I can just say "sounds good". Or maybe provide a small bit of color like: "sounds good but make sure to consider X".
Often we will have a side discussion about that particular issue until I'm satisfied. This happen more when I'm doing design / architectural / planning sessions with the AI. It can be as short or as long as it needs. And then we move on to the next one.
My main goal with these strategies is to help the AI get the relevant knowledge and expertise from my brain with as little effort as possible on my part. :D
A few other tactics:
- You can address multiple at once: "Item 3, 4, and 7 sound good, but lets work through the others together."
- Defer a discussion or issue until later: "Let's come back to item 2 or possibly save for that for a later session".
- Save the review notes / analysis / design sketch to a markdown doc to use in a future session. Or just as a reference to remember why something was done a certain way when I'm coming back to it. Can be useful to give to the AI for future related work as well.
- Send the content to a sub-agent for a detailed review and then discuss with the main agent.
IDK would require the LLM to be aware of the frequency of cases seen in its own training.
I can see this working as a risk ranking, which is certainly worth trying in its own right.
Does it actually say “I don’t know?”
There are so many overlapping and also unique approaches to software development beyond vibe coding and ai driven software development.
It would help if you briefly specified the AI you are using here. There are wildly different results between using, say, an 8B open-weights LLM and Claude Opus 4.6.
The test cases themselves becomes the foci - the LLM usually can't get them right.
Consider the the following: Unit, Integration, System, UAT, Smoke, Sanity, Regression, API Testing, Performance, Load, Stress, Soak, Scalability, Reliability, Recovery, Volume Testing, White Box Testing, Mutation Testing, SAST, Code Coverage, Control Flow, Penetration Testing, Vulnerability Scanning, DAST, Compliance (GDPR/HIPAA), Usability, Accessibility (a11y), Localization (L10n), Internationalization (i18n), A/B Testing, Chaos Engineering, Fault Injection, Disaster Recovery, Negative Testing, Fuzzing, Monkey Testing, Ad-hoc, Guerilla Testing, Error Guessing, Snapshot Testing, Pixel-Perfect Testing, Compatibility Testing, Canary Testing, Installation Testing, Alpha/Beta Testing...
...and I'm certain I've missed dozens of other test approaches.
No it is not.
There os no amount of testing that can fix a flawed design
Absolutely. Tight feedback loops are essential to coding agents and you can’t run pipelines locally.
Everyone thinks LLMs are good at the things they are bad at. In many cases they are still just giving “plausible” code that you don’t have the experience to accurately judge.
I have a lot of frontend app dev experience. Even modern tools (Claude w/Opus 4.6 and a decent Claude.md) will slip in unmaintainable slop in frontend changes. I catch cases multiple times a day in code review.
Not contradicting your broader point. Indeed, I think if you’ve spent years working on any topic, you quickly realize Claude needs human guidance for production quality code in that domain.
There’s also a big disconnect in terms of SDLC/workflow in some places. If we take at face value that writing code is now 10x faster, what about the other parts of the SDLC? Is your testing/PR process ready for 10x the velocity or is it going to fall apart?
What % of your SDLC was actually writing code? Maybe time to market is now ~18% faster because coding was previously 20% of the duration.
Even non-influencers are trying to exaggerate their LLM skills as a way to get hired or raise their status on LinkedIn. I rarely read the LinkedIn social feed but when I check mine it’s now filled with claims from people about going from idea to shipped product in N days (with a note at the bottom that they’re looking for a new job or available to consult with your company). Many of these posts come from people who were all in on crypto companies a few years ago.
The world really is changing but there’s a wave of influencers and trend followers trying to stake out their claims as leaders on this new frontier. They should be ignored if you want any realistic information.
I also think these exaggerated posts are causing a lot of people to miss out on the real progress that is happening. They see these obviously false exaggerations and think the opposite must be true, that LLMs don’t provide any benefit at all. This is creating a counter-wave of LLM deniers who think it’s just a fad that will be going away shortly. They’re diminishing in numbers but every LLM thread on HN attracts a few people who want to believe it’s all just temporary and we’re going back to the old ways in a couple years.
I was listening to an "expert" on a podcast earlier today up until the point where the interviewer asked how long his amazing new vibe-coded tooling has been in production, and the self-proclaimed expert replied "actually we have an all-hands meeting later today so I can brief the team and we will then start using the output..."
This always seems to be the pattern. "I vibe coded my product and shipped it in 96 hours!" OK, what's the product? Why haven't I heard of it? Why can't it replace the current software I'm using? So, you're looking for work? Why is nobody buying it?
Where is the Quicken replacement that was vibecoded and shipping today? Where are the vibecoded AAA games that are going to kill Fortnite? Where is the vibecoded Photoshop alternative? Heck, where is the vibecoded replacement for exim3 that I can deploy on my self hosted E-mail server? Where are all of the actual shipping vibecoded products that millions of users are using?
But even if human interaction wasn't needed, the question still stands, would you trust an LLM to calculate your tax liability just by feeding it data?
Claude Code and OpenClaw - they are vibecoded. And I believe more coming.
Also people are using CC for the cheap access to the model, otherwise they'd be using opencode.
I note that games are mostly art assets and things like level design, and players are already happy to instantly consign such products to the slop bin.
The whole thing is "market for lemons": app stores filling with dozens of indistinguishable clones of each product category will simply scare users off all of them.
https://www.reddit.com/r/selfhosted/comments/1rckopd/huntarr...
One redditor security reviews a vibe coded project
The maintainer, instead of listening to the security researcher and accepting feedback about his development process, instead:
1. Denied the problem
2. Censored discussion of the problem
3. Banned the people calling out the problem
...and then when the security issues were posted more publicly and got traction...
4. Made the subreddit private
5. Wiped and deleted his account
6. Wiped and deleted the GitHub repo
7. Took the project's web site off the web
Absolutely wild and unhinged behavior.
That said, I have had some good experiences getting a few features from zero to working via LLMs and it's helped me find lots of bugs far easier than my own looking.
I can imagine a vibe coded todo app. I can also kind of imagine a vibe coded gIMP/Photoshop though it would still take several person years, prompting through each and every feature.
With all that said, today’s LLMs do seem so provide a little bit more value compared to the bit chain thing, for example OCR/.pdf parsing is I’d say a solved thing right now thanks to LLMs, which is nice.
This is ditto my observation. There seems to be a certain "type" of people like this. And it's not just people looking for work.
My guess is either they have super low critical thinking, a very cynical view of the world where lies and exaggeration are the only way to make it, or something more pathological (narcissism etc).
I have a relative who was late to crypto, late to drop shipping, late to carbon credits, but is now absolutely all-in on AI as his ticket out. It honestly depresses the hell out of me trying to talk to him because everything is about money and getting rich.
People like this don't care about underlying technologies or learning past the most basic surface level of understanding.
For me it is far more than 10x, but I consider noobs by saying 10x instead of 20x or more.
So instead of the 10x coder doing it, the 1x coder does it, but then that factor of 3x becomes 0.3x.
A lot of people are OCD pedants about stuff that can be solved with a linter (but can’t be bothered to implement one) or just “LGTM” everything. Neither provide value or feedback to help develop other devs.
This many be one of the best quotes on HN in a while.
I can't say how many times the LLM-proposed solution to a jittery behavior is adding retries. At this point we have to be even more careful with controlling the implementation of things in the hot path.
I have to say though, giving Amp/Claude Code the Grafana MCP + read-only kubectl has saved me days worth of debugging. So there's definitely trade-offs!
That says something about how much some people care about this.
Actually I had some terrible experiences when asking the agent to do something simple in our codebase (like, rename these files and fix build scripts and dependencies) but it spent much longer time than a human, because it kept running the full CI pipelines to check the problems after every attempted change.
A human would, for example, rely on the linter to detect basic issues, run a partial build on affected targets, etc. to save the time. But the agent probably doesn't have a sense of time elapsed.
Co-pilot said something about having too many rows returned and had some complex answer on how to reduce row count.
I just added a "LIMIT 100" which was more than adequate.
LLM issues can often be solved by being more and more specific, but at some point being specific enough is just as time consuming as jumping in and doing it yourself.
> The back and forth part can be faster if e.g. you have scripts/programs that deterministically check outputs
This is where configuration language like CUE can be useful in complementing LLM [1].
It's the deterministic NLP cousin of the stochastic LLM based on mathematically sound latticed-value logic [2].
[1] Guardrailing Intuition: Towards Reliable AI:
https://cue.dev/blog/guardrailing-intuition-towards-reliable...
[2] The Logic of CUE:
This makes sense when you stop viewing the LLM as a "vending machine" for apps and start seeing it as a repository of software deltas.
LLMs aren't just trained on final code; they are trained on the entire history of pull requests, review comments, and issue discussions that move a project from one version to the next.
When I use an LLM now, my workflow has shifted entirely. I’ve stopped trying to be the "coder" and have instead stepped into the role of PR Reviewer and Power User. My job is to point out edge cases, define the spec, and catch regressions—effectively managing a "virtual team" that handles the boilerplate and feature implementation.
Expecting a one-shot 1.0 release is unrealistic because it bypasses the thousand micro-decisions that happen in a real dev cycle. By embracing the "review and refine" loop, I’m becoming a better maintainer, even if that 100-hour gap to a polished product still exists.
I know it's not the point of this article, but really?
It could have been a web app, but with NFTs and Farcaster miniapps, you market to people who are willing and able to spend using their wallet instead of asking “normies” for credit card information for a 2 dollar custom image (that you could also prompt out of a free Gemini session).
With Farcaster, you also already have the profile picture of the user, one less hurdle again.
AI can be really attractive to bros but also be incredibly useful.
In other words, AI isn't a trend that's going to pass, it's permanently going to reshape the tech scene and economy in a way that cryptocoins and NFTs absolutely did not.
This exact wording was used for crypto. "It isn't a trend that's going to pass" and "It's going to reshape everything." Why are we sure of it now for AI (and that we're going to be right), when they were also sure of it before for crypto (and they ended up wrong)?
The AI people have the exact same feelings of absolute certainty as the crypto people had.
Probably zero grandmothers outside the west, and very few grandmothers within the west, know what NFT even stands for.
Yes, this is what hype is like. Something being hyped doesn't automatically mean it's going to go bust.
Again, everyone smart was making fun of NFT at its peak. People who accomplish things are excited by LLMs, not just teenagers, grifters, bros, and your weird uncle.
Are people just expecting there's going to be enough digital fools to make a market?
A movie can be duplicated indefinitely. There's no guarantee your song will be appreciated as art. I'm not sure why you say you can't print out an image and hang it in your living room; we do that all the time at home.
I've personally never dabbled in NFTs, but I don't think it's fair to ascribe the inherent conflict between information and scarcity uniquely to them.
It's bunk.
There's some 80-20:ness to all programming, but with current state of the art coding models, the distribution is the most extreme it's ever been.
Honestly, seeing all the dumb code that it produces, calling this thing "intelligent" is rather generous...
I suppose if you are doing something that truly can be decided based on a test but, I just don't see it, at least for anything I do.
If I'm reviewing all the code, so far I'm still the bottleneck even with a single agent and I don't see an easy way to change that.
When we start selling the software, and asking people to pay for/depend upon our product, the rules change -substantially.
Whenever we take a class or see a demo, they always use carefully curated examples, to make whatever they are teaching, seem absurdly simple. That's what you are seeing, when folks demonstrate how "easy" some new tech is.
A couple of days ago, I visited a friend's office. He runs an Internet Tech company, that builds sites, does SEO, does hosting, provides miscellaneous tech services, etc.
He was going absolutely nuts with OpenClaw. He was demonstrating basically rewiring his entire company, with it. He was really excited.
On my way out, I quietly dropped by the desk of his #2; a competent, sober young lady that I respect a lot, and whispered "Make sure you back things up."
I needed it, I quickly build it myself for myself, and for myself only.
Also though, I feel like being attached to Confluence helped it because there is a lot less competition in the world of documentation wikis than there is in task management.
One of the most valuable things about Slack is the ecosystem: apps, API support, etc. If you need to receive notifications from external apps (like PageDuty or Incident.io or something like that), good luck expecting them having a setup for your own version of the app. Yeah, some of them provide webhooks (not all of them), but in the end you have to maintain that too...
It was: "With sufficiently advanced vibe coding the need for certain type of product just vanishes."
If a product has 100 thousand users and 1% of them vibe codes an alternative for themselves, the product / business doesn't vanish. They still have 99 thousand of users.
That was the rebuttal, even if not presented as persuasively and intelligently as I just did.
So no, it's not the case of "both things being true". It's a case of: he was wrong.
You can learn to bake good bread. It’s not _that_ hard. And it’ll probably taste better than store bought bread.
But it almost certainly won’t be cheaper. And it’ll take a more more time and effort.
Still, sometimes you might bake your own bread for kicks. But most of the time, you’ll just buy the bread someone else has already perfected.
I can have fresh bread anytime I want from a handful of nearby stores.
Could you do the same in eg. Photoshop? Maybe, but even if, you would need to learn how.
I tried it works wells. I can do the same thing in my Linux machine, but even my 12 year old now can get perplexity to build him a tool to compare ram prices at different chinease vendors.
You really believe that?
Unless LLMs can read minds, no one will bother to specify, even in plain english with the required level of detail. And that is assuming the user has the details in mind, which is also something pretty improbable...
The average user just has no interest in building things.
There are well attended parent evenings in our school on that topic.
Thinking about it, we should turn these into vibe coding hackathons where we replace all the ad-ridden little games, learning tools, messengers we don't like with healthy alternatives.
That would be good news, but I doubt most people will do things like that.
What about this is new?
Sitting down with a child to teach them the very basics of javascript in an hour? Trivial.
Needing Claude to do it is kind of embarassing, if anything.
This is not the future of software.
See here if you're interested: stefansiegert.net/cube-timer
Let me know if you adapt it in any way, my son would be delighted to see open source work its magic :)
The things that are going away are tools that provide convenience on top of a workflow that's commoditized. Anything where the commercial offering provides convenience rather than capabilities over the open source offerings is gonna get toasted.
An example would be if the user only ever works with .jpg files, then you don't need to support any of the dozens of other formats an image program would support.
I cannot stress enough how many software users out there are only using 1-10% of a program's capability, yet they have to pay for a team of devs who maintain 100% of it.
They just want the software to do the few things they need it to do. AI labs are falling over themselves to remove the gate keeping regular people from using their computing device the way they want to use it. And the progress there in the last few years is nothing short of absolutely astounding.
Yet, all the astounding progress notwithstanding, I don't have a suite of bespoke tools replacing the ones I depend on. I cannot say "hey claude, make me a suite of bespoke software infrastructure monitoring and operational tooling tailored to my specific needs" and expect anything more than a giant headache and wasted time. So maybe we just need to wait? Or maybe it's just not actually real. My view is unless you show me a working demo it's vaporware. Show me that the problem is solved, don't tell me that it might be solved later sometime.
I could certainly imagine building myself some sort of dashboard. It would seem like a prime use case.
You want to hear about a problem solved? Recently I extended a tool that snaps high resolution images to a Pixel art grid, adding a GUI. I added features to remove the background, to slice individual assets out of it automatically, and to tile them in 9-slice mode.
Could I have realistically implemented the same bespoke tool before AI? No.
Let's say I emit roughly 1TB of telemetry data per day--logs, metrics, etc. That's roughly what you might expect from medium sized tech company or a specific department (say, security) at a large company. There is going to be a significant infrastructure investment to replicate datadog's function in my organization, even if I only use a small subset of their product. It's not just "building a dashboard" it's building all the infrastructure to collect, normalize, store, and retrieve the data to even be able to draw that dashboard.
The dashboard is the trivial part. The hard part is building, operating, and maintaining all the infrastructure. Claude doesn't do a very good job helping with this, and in some sense it actually hinders.
EDIT: I'm not saying you shouldn't take ownership of your telemetry data. I think that's a strategically (and potentially from a user's perspective) better end result. But it is a mistake to trivialize the effort of that undertaking. Claude is not going to vibeslop it for you.
For my use case I wanted bespoke software to work with Pixel art, but obviously I would not try to build Photoshop or Aseprite from scratch. I needed only specific functionality and I was able to build that in a way fitting my workflow better than any existing software could.
I was able to build it with Claude Code and Codex. Maybe the implementation is sloppy, I did not care to check. The program works, and it's like a side project to my side project. It would not have been possible in the past, I would have needed to work with what Aseprite offers out of the box.
LLM's change the calculus of the above chart dramatically.
I've always said, the easiest part of building software is "making something work." The hardest part is building software that can sustain many iterations of development. This requires abstracting things out appropriately which LLMs are only moderately decent at and most vibe coders are horrible at. Great software engineers can architect a system and then prompt an LLM to build out various components of the system and create a sustainable codebase. This takes time an attention in a world of vibe coders that are less and less inclined to give their vibe coded products the attention they deserve.
It has examples of all the parts of a web app written, over many years, to be my own ideal structure. When the LLM era arrived, I added a ton of comments explaining what, why and how.
It turns out to serves as a sort of seed crystal for decent code. Though, if I do not remind it to mimic that architecture, it sometimes doesn't and that's very weird.
Still, that's a tip I suggest. Give it examples of good code that are commented to explain why its good.
The difference I've noticed is that the act of actually typing out code made me backtrack a few times refining the possible solutions before even starting the integration tests, sometimes before even doing a compile.
When generating, the LLM never backtracked, even in the face of broken tests. It would proceed to continue band-aiding until everything passed. It would add special exceptions to general code instead of determining that the general rule should be refined or changed.
The reason that some devs are reporting 10x productivity is because a bunch of duct-taped, band-aided, instant-legacy code is acceptable. Others who dont see that level of productivity increase are spending time fixing the code to be something they can read.
Not sure yet if accepting the spaghetti is the right course. If future LLMs can understand this spaghetti then theres no point in good code. If we still need human coders, then the productivity increase is very small.
That is pretty bad..
Trello was written by interns as a summer project, when SPAs were just becoming a thing and React didn't even exist.
With 30 hours I bet I could get a pretty good one up without vibe coding it.
In a single afternoon I could get boards, cards, lanes, etc done. React, MaterialUI uaing Grid + Card and you're almost done.
It sped me up (and genuinely helped with some ideas) but not 10x.
The bits I didn't design myself I definitely needed to inspect and improve before the ever eager busy beaver drove them to the ground.
That said, I'm definitely impressed by how a frontier model can "reason" about Go code that's building an AST to generate other Go code, and clearly separate what's available at generation time vs. at runtime. There's some sophistication there, and I found myself telling them often "this is the kind of code I want to generate, build the AST."
I also appreciated how faster models are good enough at slightly fuzzy find and replace. Like I need to do this refactor, I did two samples of it here, can you do these other 400? I have these test cases in language X, converted 2, can you do the other 100? Even these simple things saved me a lot of time.
In return I got something that can translate SQLite compiled to Wasm into 500k lines of Go in about a month of my spare time.
those are not copies, they aren't even features. usually part of a tiny feature that barely works only in demo.
with all vibe coding in the world today you still need at least 6 months full time to build a nice note taking app.
If we are talking something more difficult - it will be years - or you will need a team and it will still take a long time.
Everything less will result in an unusable product that works only for demo and has 80% churn.
It depends entirely on what you want. You can literally code a JavaScript 1-liner that will make a <textarea> then put the content back in the URL and it will work serverless on pretty much any platform with a Web browser.
You can also write a note taking app that will be federated yet private, that will have its own scripting language, etc. I mean you can yak-shave your way to write your own OS or even designing your own CPU for that.
So... I'm not sure that metric, time, means much without a proper context, including who does it. It's quite different if to do that, regardless of the tooling used, if you are a professional developer, designer, fullstack dev, prototypist, PM, marketer, writer, etc.
sure. does your note taking app supports formatting? you don't need it today. you will need it at some point. images? same.
does it handle file corruption etc? no? then its pretty much useless.
does it work across devices? in modern world, again, it is pretty much useless without it
it works across devices? then it needs hosting. if it is hosted it needs auth, it needs backups
you can go on for ever.
the bar for very minimal note taking app that you actually will use is very high, with other software it is even higher.
and this is not even state of art, this is must haves
And even so if your starting a note taking app most of those problems like file corruption and image support are largely solved problems. There is also the benefit of being able to reference tons of open source implementations.
I think one month to notion like app that is prod ready if you just need Auth + markdown + images + standard text editing
When everyone is able to make their own one off prototype in 30 minutes, no one will pay for the thing that took someone 6 months.
there is very very rare use case when diy makes sense. in 99% of cases its just a toy that feels nice as you kinda did it. but if you factor in the time etc it is always costs 100x more than $5/month you could usually buy
In late 2024 it might have taken 6 months. Today, two weeks, maybe 3.
Bad example, note apps loaded with features are anti-productive and are for people who treat note taking as a hobby itself.
You have Obsidian anyway if you want something open source to work with.
The old rules still apply mainly.
This is in-line with Googles study showing about a 10% productivity increase and other research I’ve read. I suspect this will increase with more integrations and workflow adaptations.
But even after power tools changed how quickly carpenters can frame and rough-in a house, the finishing work (which uses power tools too) still takes the majority of the time.
The details and pitfalls that are unique to your specific scenario, that you only discover by running into them.
And yet this less obvious, more uncommon stuff is also what AI will be weakest at.
Also this article uses 'pfp' like it's a word, I can't figure out what it means.
I'm able to vibe code simple apps in 30 minutes, polish it in four hours and now I've been enjoying it for 2 months.
My prompt "What does PFP mean on this page: https://kanfa.macbudkowski.com/vibecoding-cryptosaurus" gave a good answer and it described extra relevant context within crypto.
I had less luck with "What does sharp tails mean in «HFT. You want low deterministic latency with sharp tails»". But I suspect the source sentence is the problem.
The result worked but that's just a hacked together prototype. I showed it to a few people back then and they said I should turn it into a real app.
To turn it into a full multi user scaleable product... I'm still at it a year later. Turns out it's really hard!
I look at the comments about weekend apps. And I have some of those too, but to create a real actual valuable bug free MVP. It takes work no matter what you do.
Sure, I can build apps way faster now. I spent months learning how to use ai. I did a refactor back in may that was a disaster. The models back then were markedly worse and it rewrote my app effectively destroying it. I sat at my desk for 12 hours a day for 2 weeks trying to unpick that mess.
Since December things have definitely gotten better. I can run an agent up to 8 hours unattended, testing every little thing and produce working code quite often.
But there is still a long way to go to produce quality.
Most of the reason it's taking this long is that the agent can't solve the design and infra problems on its own. I end up going down one path, realising there is another way and backtracking. If I accepted everything the ai wanted, then finishing would be impossible.
Hasn't happened in a long time. Opus 4.6 is a miracle improvement.
So many people are just shouting ‘I wanna go fast’ and completely forgetting the lessons learned over the past few decades. Something is going to crash and burn, eventually.
I say this as a daily LLM user, albeit a user with a very skeptical view of anything the LLM puts in front of me.
"when used appropriately" means:
- Setting up guardrails: use a statically typed language, linters, CLAUDE.md/skills for best practices.
- Told to do research when making technical decisions, e.g. "look online for prior art" or "do research and compare libraries for X"
- Told to prioritize quality and maintainability over speed. Saying we have no deadline, no budget, etc.
- Given extensive documentation for any libraries/APIs it is using. Usually I will do this as a pre-processing step, e.g. "look at 50 pages of docs for Y and distill it into a skill"
- Given feedback loops to check its work
- Has external systems constraining it from making shortcuts, e.g. "ratchet" checks to make sure it can't add lint suppressions, `unsafe` blocks, etc.
And, the most important things:
- An operator who knows how to write good code. You aren't going to get a good UI/app unless you can tell it what that means. E.g. telling it to prioritize native HTML/CSS over JS, avoiding complexity like Redux, adding animations but focus on usability, make sure the UI is accessible, etc.
- An operator who is steering it to produce a good plan. Not only to make sure that you are building the right thing, but also you are explaining how to test it, other properties it should have (monitoring/observability, latency, availability, etc.)
A lot of this comes down to "put the right things in the context/plan". If you aren't doing that, then of course you're going to get bad output from an LLM. Just like you would get bad output from a dev if you said "build me X" without further elaboration.
Why would it become a mess? If I made a skill for, say, BuildKite, Claude will only load it when applicable.
They will eventually get out of date, so that is something you’ll have to accept or perhaps make some meta skill-updater program to run on a schedule
(emphasis added)
Not sure if it was actually written by hand or AI was glossed over, but as soon as giving away money was on the table, the author seems to have ditched AI.
Expecting a one-shot 1.0 release is unrealistic because the sheer volume of context and decision-making required for a finished product is enormous.
Instead, I think of LLMs as being trained on the "delta" of software development: the pull requests, review comments, and issue discussions that move a project from one version to the next.
When you use an LLM for coding, you are effectively tapping into the collective output of a team of developers and a crowd of users. My mental model has shifted accordingly: I no longer try to be the "coder." Instead, I act as the PR reviewer and the passionate power user. My job is to point out edge cases and refine the output, rather than expecting a finished product in one go.
It’s making me a better maintainer and a more precise communicator, even if the "100-hour gap" to production remains a reality.
To have a polished software project, you must spend time somewhat menially iterating and refining (as each type of user).
To have a polished software project, you need to have started with tests and test coverage from the start for the UI, too.
Writing tests later is not as good.
I have taken a number of projects from a sloppy vibe coded prototype to 100% test coverage. Modern coding llm agents are good at writing just enough tests for 100% coverage.
But 100% test coverage doesn't mean that it's quality software, that it's fuzzed, or that it's formally verified.
Quality software requires extensive manual testing, iteration, and revision.
I haven't even reviewed this specific project; it's possible that the author developed a quality (CLI?) UI without e2e tests in so much time?
Was the process for this more like "vibe coding" or "pair programming with an LLM"?
A 40-hour work year has 2,080 hours per person per year.
The "10,000" hours necessary to be really good at anything number was the expert threshold that they used to categorize test subjects who performed neuroimaging studies while compassion meditating. "10,000" hours to be an expert is about 5 years at full time.
But how many hours to have a good software product?
Usually I check for tests and test coverage first. You could have spent 1,000 hours on a software project and if it doesn't have automated tests, we can't evolve the software and be sure that we haven't caused regressions.
Again, I haven't even read this particular project;
There's:
Prompt insufficiency: Was the specification used to prompt the model to develop the software sufficient in relation to what are regarded as a complete enough software specifications?
Model and/or Agent insufficiency,
Software Development methods and/or Project Management insufficiency,
QA insufficiency,
Peer review sufficiency;
Is it already time to rewrite the product using the current project as a more sufficient specification?
But then how many hours of UI and business logic review would be necessary again?
And then there is one guy, a friend of mine, who is planning to release a "submit a bug report, we will fix it immediately" feature (so, collect error report from a user, possibly interview them, then assess if its a bug or not with a "product owner LLM", and then autonomously do it, and if it passes the tests - merge and push to prod - all under one hour. Thats for a mid cap company, for their client-facing product. F*** hell! I have a full bag of bug reports ready for when this hits prod :->
This is a bit of an unknown right now. If you get a working prototype, but need to productionize it, make sure it can scale, and get it looked over with a security mineset, how long it might take isn't clear, so finding someone who will do that is hard.
I built 4 AI products to hundreds of thousands of users, working with AI agents as collaborators, not autopilots. The difference isn't the tool. It's whether you can tell the AI is wrong and stop it before it wastes 10 hours going down the wrong path.
The author watched Claude create new S3 buckets for several rounds before catching it. An experienced engineer catches that on the first diff. Most of those 100 hours were spent not knowing you're lost.
"Vibecoding" as a concept is the problem. It implies you can vibe your way through engineering. You can't. AI is a force multiplier, not a replacement for knowing what good looks like.
Not knocking the premise of the post. It probably works well for one single user if it’s an iPhone or Android app. But his 100 power hours are probably just right for what he ended up launching as he iterated through the requirements and learned how to set this up through reinforced learning and user feedback.
I find that vibe coding is useful when it can be build with little details and it makes the right assumptions.
Used Codex for the whole project. At first I used claude for the architect of the backend since thats where I usually work and got experience in. The code runner and API endpoints were easy to create for the first prototype. But then it got to the UI and here's where sh1t got real. The first UI was in react though I had specifically told it to use Vue. The code editor and output window were a mess in terms of height, there was too much space between the editor and the output window and no matter how much time I spent prompting it and explaining to it, it just never got it right. Got tired and opened figma, used it to refine it to what I wanted. Shared the code it generated to github, cloned the code locally then told codex to copy the design and finally it got it right.
Then came the hosting where I wanted the code runner endpoint to be in a docker container for security purpose since someone could execute malicious code that took over the server if I just hosted it without some protection and here it kept selecting out of date docker images. Had to manually guide it again on what I needed. Finally deployed and got it working especially with a domain name. Shared it with a few friends and they suggested some UI fixes which took some time.
For the runner security hardening I used Deepseek and claude to generate a list of code that I could run to show potential issues and despite codex showing all was fine, was able to uncover a number of issues then here is where it got weird, it started arguing with me despite showing all the issues present. So I compiled all the issues in one document, shared the dockerfile and linux secomp config tile with claude and the also issues document. It gave me a list of fixes for the docker file to help with security hardening which I shared back with codex and that's when it fixed them.
Currently most of the issues were resolved but the whole process took me a whole week and I am still not yet done, was working most evenings. So I agree that you cannot create a usable product used by lots of users in 30 minutes not unless it's some static website. It's too much work of constant testing and iteration.
It has basically eliminated surprises like that.
EXCEPT... you've just vibe coded the first 90 percent of the product, so completing the remaining 10 percent will take WAY longer than normal because the developers have to work with spaghetti mess.
And right there this guy has shown exactly how little people who are not software developers with experience understand about building software.
I expected OP to actually 'learn' devops, but what they did was just asking LLMs to do everything.
Also...
> 180+ paid $2 for a dino
People pays $2 for an image of dinosaur with human face?
Some people seem to be better at it than others. I see a huge gulf in what people can do. Oddly there is a correlation between was a good engineer pre AI and can vibe code well.
But I see one odd thing. A subset of those who people would consider good or even amazing pre AI struggle. The best I can tell at this stage is because they lacked get int good results with unskilled workers in the past and just relied on their own skills to carry the project.
AI coders can do some amazing things. But at this stage you have to be careful about how you guide it down a path in the same way you did with junior engineers. I am not making a comparison to AI being junior, they by far can code better than most senior engineers, and have access to knowledge at lighting speed.
There are some good points here to improve harnesses around development and deployment though, like a deployment agent should ask if there is an existing S3 bucket instead of assuming it has to set everything up. Deployment these days is unnecessarily complicated in general, IMO.
Even pretty massive companies like databricks don't think about those things and basically have a UI template library that they then compose all their interfaces from. Nothing fancy. Its all about features, and LLM create copious amounts of features.
As we move from tailors to big box stores I think we have to get used to getting what we get, rather than feeling we can nitpick every single detail.
I'd also be more interested in how his 3rd, 4th or 5th vibe coded app goes.
If you're asking the AI to generate large amounts of code that you don't really look at, you're definitely vibecoding. If you are mainly writing the code yourself with some assistance from AI tools, you are not vibecoding.
Naturally it's fuzzy in the middle. And for people without software development skills, vibecoding is the only option they have.
I would find it a bit tricky to write a full test suite for a product without any code though. You'd need to understand the architecture a bit and likely end up assuming, or mocking, what helpers, classes, config, etc will be built.
I’m working on project - a password manager, where I have full end to end test harnesses - cli client makes changes, sync them to the server and then observe the data in iOS app running in the emulator. More than once I noticed codex just hard coded expected values from the test harnesses directly into UI layout in iOS app to make the test pass…
Similar issues in the crypto layer - tests were written first , then code was written . During the review I noticed that the code was made to just pass the test - the logic was to check if signature values exists instead of checking if crypto signature is valid.
LLM can help with code reviews as well, but it has to be guided specifically what to look for for. This is with codex 5.4 model
Something much closer to production SDLC patterns than a Figma mockup.
The interesting shift seems to be that building the first version is no longer the bottleneck — distribution, UX polish and reliability are.
Great vibe coded projects optimize for simple, legible, and complete.
Everything else regresses to a toy.
I have to say its a little sad that so many devs think of security and cryptography in the same way as library frameworks. In that they see it as just some black box API to use for their projects rather than respecting that its a fully developed, complex field that demands expertise to avoid mistakes.
Curious. Can you elaborate on this a bit?
There are plenty of ways to code and use code, which-ever works for you is good just improve on it and make it more effective. I have multiple screens on my computer, i don't like jumping back and fourth opening tabs and browsers so i have my set up the best way that works for me. As for the AI models, they are not going to be that helpful to you if you don't understand why its doing what its doing in a particular function or crate (in case of rust) or library. I imagine the the over the top coder that has years of experience and multiple knowledge in various languages and depth knowledge in libraries, using the same technique he can replace a whole Department by himself technically.
The only thing he needed to code was an NFT wrapper, which presumably is just forking an existing NFT wholesale.
The interesting, user-facing part of the project isn't code at all! It's just an HTML front end on someone else's image generator and a "pay me" button.
Very disappointing.
I would say the remaining 10% are about how robust your solution is - anything associated with 'vibe' feels inherently unsecure. If you can objectively proof it is not, that's 10 % time well spend.
Only "feels"?
Which part of "commodity" is confusing???
When an agent takes a shortcut early on, the next step doesn't know it was a shortcut. It just builds on whatever it was handed. And then the step after that does the same thing. So by hour 80 you're sitting there trying to fix what looks like a UI bug and you realize the actual problem is three layers back. You're not doing the "hard 20%." You're paying interest on shortcuts you didn't even know were taken. (As I type this I'm having flashbacks to helping my kid build lego sets.)
The author figured this out by accident. He stopped prompting and opened Figma to design what he actually wanted. That's the move. He broke the chain before the next stage could build on it. The 100 hours is what it costs when you don't do that.
Apparently LLMs break some devs brains though. Because it's not one shot perfect they throw their hands in the air claim AI can't ever do it and move on, forgetting all those skills they (hopefully) spent years building to manage complex software. Of course a newbie vibe coder won't know this but an experienced developer should.
I can look at code I wrote years ago and have absolutely no memory of writing it, but I know its my code and I know where some of the warts and traps are. I can answer questions about why things work a certain way.
With an LLM, you don't get that. You're basically starting from scratch when it comes to solving any problem or answering any question.
Best part, it's good for people too.
Ok clanker
[Disclaimer: that I have read. Doesn't mean there weren't others.]
Too bad it's about NFTs but we can't have everything, can we?