If AI can write better code and design better architecture, why can’t it make better product decisions? I don’t see how they are fundamentally different
1. Product decisions require far more knowledge than coding and architecture decisions, and that knowledge is harder to come by. It often requires research, talking to customers, understanding the organization of the company, the company's mission, the company's employees, their performance, stakeholders, the competitive landscape, funding, growth/sales/marketing strategies and budgets, customer buying patterns, company strategy, etc. Not to mention the code, the coding backlog, etc. Companies have to compete in a hostile, competitive, capitalistic market, which means they're trying to beat other companies, which means a seemingly very good product decision can actually be bad solely because other people are doing it better. Imagine if your perfectly functional code failed to compile simply by nature of the fact that better programmers exist.
2. Product design requires taste, and taste is partly a function of novelty, but there aren't that many LLMs, so everyone's using the same few LLMs, therefore everyone's getting the same designs, therefore the designs look like clones of each other rather than novel thoughtful design, therefore they fail the taste bar. In addition, LLMs simply weren't trained on good vs. bad visual or product design nearly as much as they've been trained on good vs. bad, functional vs. non-functional code.
With code, it's fairly easy to have unit or end to end tests. That can be relatively objective and deterministic, and pretty quick.
Architecture is one layer removed, and far harder to test. You're stuck with things that are at least an order of magnitude to prove autonomously; will it scale in terms of performance, will the abstractions be a good fit for future requirements, etc. Testable, but much harder.
Design; well, there's possibly some metrics by which you could measure UX (attention for the viewer, contrast ratios, number of actions needed to achieve something, how far the mouse had to travel), but really the majority of it is subjective. "How does the human feel when viewing this" - I suspect almost by definition needs either a human, or a very good model of it. I don't think we have the latter, making this pretty hard.
You're relying on that information not being in the training data, because you are working with third-party generic models, so your expertise is safe.
Also, where are the AI-operated startups?? Why has nobody’s openclaw biz gone multinational by now?
Maybe it keeps deleting and the emails and business records.
https://www.pcmag.com/news/meta-security-researchers-opencla...
Seems like it happens fairly often, considering the amount of pages like this I just found.
https://drecov.pandaoffice.com/data-recovery/how-to-fix-open...
Also, LLMs in these apps use a lot of tokens, so people often use lower-end models, which increases the probability of this happening.
AI is not capable of empathy.
AI is not capable of having good taste.
I have an experiment to offer. Find a really, really great example of something poorly-designed, too busy or whatever other metric you want to use as your "poorly-designed" thing, then ask Claude what's wrong with it and how it should be improved?
Because I can imagine how that conversation would go with most of the designers I've worked with.
This is like explaining to someone that there are no little people in the TV set, what a movie is, and then they go "yeah but sometimes they don't just talk, they also sneeze, and you can't fake sneezing, I read that in an article".
Training data gives the LLM it's knowledge of the world, it can then bisect that space and output most likely answers.
If you are able to write a prompt such that you get the LLM to bisect the training data space, you will get the output that you want.
The trick though here is that guessing what is a profitable business venture is to some extent the same as trying to predict the future.
However, I would assume, that as more and more people start living more of their life via LLMs (i.e. asking where should I go for vacation, what are good shoes to buy, which is the most popular sports team near me) consumer choices will start to become more homogenous which would make it easier and more likely that for businesses to succeed they have to follow the LLM advice as that is also what the consumers are doing.
Geopolitical decisions involve pursuing outcomes in situations involving far more variables, often changing in realtime due to factors outside the "game board", where the rules are complex, mutable, and not necessarily known in advance, and where the success and failure conditions are subjective and may themselves change during the course of the scenario.
Geopolitical situations are Chess, Nomic, and Werewolf all rolled into one. How good is AI at winning all of these games?
There’s no need to address it literally.)
I hope we'll have some kind of good paradigm to do lora style learning or maybe an output subagent or something so we can better calibrate the output style in a fuzzy way. In context learning (instructions, memories) doesn't currently seem that effective.
> But I suspect it goes deeper. Why does it seem to begin sentences with "Additionally" a bit too much? My guess is that it's because humans would tend to base their writing much more on, I don't know, maybe an underlying narrative structure of sorts, such that the relevance of things and the connections between them can more often be implied.
> Meanwhile AI has little or none of this kind of context, so it needs to use a lot more explicit connectors, and it's forced to risk being mildly patronizing for lack of a nuanced reader persona to write for. It doesn't encapsulate opinion in the subtext and details of factual statements, there is nothing to be found between the lines, and no arcane divination is needed to see what the angle is.
Scattered throughout the section before that (by which I mean that the next quotes weren't contiguous) is some tracing around the core problem that I later realized is a lack of qualia. Is it that the bot has no capacity for qualia? I mean, I would think so. I'm not sure if that's strictly compatible with the concept of qualia, since I'm under the impression that they're supposed to be ineffable anyway, so how would we really know? But either way, it obviously lacks direct knowledge of the qualia of a lot of things it pretends to have experienced. This also shows in its attempts at metaphor and analogy (but bear in mind this is all from early 2025).
> You might also find some detached or glossed-over metaphors. Is this "[I'm just a normal person, not some] superhero with a secret morning routine" really a particularly meaningful concept to you? It feels empty to me. Who has a "secret morning routine"? Who would have been thinking of such a thing before it was forced into the frame by the writer? There will often be a lot of this kind of disingenuous pretending at human experience.
> The scenario . . . where someone is having fully articulated, coherent, and civil self-talk about their alarm while it is still going off, seems unrealistic to me.
> Oh, you sat wrapped in a hoodie with the morning's hot drink to watch the sunrise? Were you also cradling the mug in both hands? It's a good thing you included such a relatable and human detail by dropping in a reference to "that soft morning light". . . . Thanks!
> "It felt kinda magical in a chill way." Seriously?
> "It's like my brain worked better without all the usual distractions." If you pause again and contemplate this bit, as well as the one from earlier about the "texts" and "chaos" that used to make his mornings so hectic, is it not empty? . . . It's alluding to things without actually talking about them. It is using many words to say nothing. It's essential to be able to notice this. --
If it was one or two paragraphs, maybe. But a full article is very tiring to read and almost always goes nowhere or goes round-and-round.
If you think your job is safe because of organizational knowledge not imbued into a commercial, third party LLM, you can be sure some billionaire and his cronies ar e racking their brains on how to reduce the number of people who do your job.