2,285 karma · joined December 10, 2011
These systems are not "intelligent" if you follow the dictionary definition. Hence the question posed to get a better understanding of how it is being used in this context.
They also do not "extract intent". There is for sure some intent behind your input to the service. What follows is a predictive text that uses your input, together with a LLM trained on a corpus with similar relations, that ultimately gives you a series of words.
That isn't to say a service like this cannot be useful. But I'm often wondering if the people who rely on these, and are particularly enthused by them, are actually aware that the terms they used are in fact anthropomorphized. I start by giving the benefit of the doubt, but it rarely lasts. 'Reasoning', 'agent', 'skill' 'hallucinate', 'know', 'think', 'train', 'learn', 'understand', 'harness', 'attention', 'context', 'prompt'.
> [Opus 5's output] is beyond the comprehension of virtually all engineers and developers
That would make it pretty bad? The key defining quality of good software, is clarity, and the ability to simplify a complex problem to the point of it seeming trivial.
> Math, science, and engineering are all contained in one model. We may be experts in one field. The model is an expert in everything that humans know.
The bar here should absolutely be to judge this against the expert level within each domain. I have time and time come across LLM output being woefully underwhelming in every single request where I am an expert. For all areas that I am not, it sure seems plausible. It is far more likely than not, that it is equally inadequate in the areas I lack the necessary knowledge to tell.
If the AI is being subpar in every field and category compared to an expert in said respective field, then, what a strange gauge of a tool's usefulness. Are we attributing higher value because a single model is "attempting to solve all knowledge and fields at the same time", why is that of any importance, or excuse?
We should not define "intelligence" as how effectively it can convince a non-expert of something being plausible. That sounds like the absolute worst tradeoff. You'd have to waste the experts time in filtering and refuting incorrect postulations that are cheep to generate. The perfect storm for bullshit asymmetry.
The reason why a PO can explain something in English, and you get something useful out at the other end (of the developer), is because of a myriad of other decisions you don't see. The reason why some software systems end up being efficient in maintenance and further development, is because of these myriad of other decisions.
The many decisions are the "devil in the details" that LLMs don't get right. Or, let's not anthropomorphize unnecessarily -- LLMs don't know right from wrong, and don't reason or reflect. They could only get this right by sheer luck. In a big numbers game, they'll always get it wrong. If you want to be a PO (or vibe coder, etc) and use English language on one end, and get these details right, there is only one possible approach:
A tight loop with expert knowledge reviewer. The programmer that knows pretty much what they want, in a small section. A LLM can draft it out so that the programmer saves time typing. This isn't really useful for the PO. (PS: The same general advice applies for any other use of LLMs. Tight loop. Expert reviewer)
You'd need a language that can express important details otherwise lost to the English language. And you'd need this to be deterministic. The "myriad of tiny decisions" are the true basis for the code implementation. If they're not expressible in the English language, and they're not achievable by LLMs, there really isn't any other way to achieve them.
First of all, private companies shouldn't be given that responsibility to begin with. Meta in particular, has a long history of unethical and immoral usage of personal data. I won't use the term "illegal", as the question of legality becomes moot when punishment can be factored in as a cost of doing business [1]. Given the long list of things Meta has been caught doing, together with the in grand total zero seconds of jail time. I'm genuinely curious as to why you think this would be any different. I'd be surprised if it hasn't already happened, where in some room without windows and a lot of lawyers and business analysts, they have ran models and concluded that the cost of getting caught here is "a good financial decision". Wouldn't be surprised either if it also came with a guarantee of personal protection from prosecution, from NSA and other government entities, in exchange for a hand in that data pipe.
Secondly, for this to carry any plausibility for being motivated by "protect the children" arguments, it requires a minimal effort be enacted on more effective measures, and a measured balance with the cost this comes at. There are very good arguments for why this law would actively harm children. Throw in some Bayesian understanding, and you better have a state of the art system that somehow pretty much never has false positives, nor false negatives, where this was also the only way to detect and avoid said abuse. I don't know the numbers here, but I highly doubt this is a good idea, even with infinite generosity as to good intentions. We've all been children, we've all done stupid things. Now throw in the brilliant and surely-not-to-scar-a-child-for-life situations where parents and strangers looking at something they thought was private, and have a "grown up discussion" about. I shiver at the thought.
Thirdly, and aside from directly harming children in situations where they selves use technology and naively, and unwisely share pictures, consider how many take pictures of their own kids without clothes, because they are normal human beings, who do not consider there to be anything sexual about said depiction. You want to throw law enforcement in the mix here? Child protective services?
Fourthly, consider the possible negative for this abuse. If normal behavior (e.g. children being children, and e.g. normal parents otherwise sharing normal pictures if you are a normal person) can be selectively chosen as being a heinous crime, this should scare anyone, especially consider the political shifting trends towards fascism.
[1]: https://www.creativefuture.org/facebook-scandal-timeline/
I dislike AI because the result has consistently been bad. The most enthusiastic AI co-workers are producing garbage at a 100x rate, while the non-enthusiastic responsible ones are left reading and reviewing it.
The article seems to make some fairly confusing statements. Why is the bar higher for software engineering, than that of civil engineering otherwise? Statements such as:
> "there is no objective reality inside software"
> "if there are many solutions to the same problem, which one is "better"?"
Is the exact same subjective goal that a objective engineering constraint has in any other engineering field. There are many ways to design and build a bridge, but the engineering aspect of it needs to model reality and account for it in such a way that the bridge to build conforms to said requirements, in a provable way. That's why engineers can be held responsible when mistakes are made.
Software Engineering can be done in the same way. This, however, depends entirely on the culture. My first decade in the field, I was fortunate to only be exposed to en environment and culture that developed software in a provably correct way, or at the very least, aspired to. The latter decade, not so fortunate. With the advent of generative AI, it's become far worse. The challenge is to carve out enough space outside the purview of "management" that wants problems solved with particular tools, regardless of applicability to said problems, and it's becoming insurmountable. Signal to noise disappearing. The idea of buying land and tending to a farm, evermore appealing.
I wonder if the author perhaps has not worked on software that comes with actual engineering constraints. There are plenty of software systems where <if it doesn't work as it should>, people die.
That isn't to say that there aren't cases where "just do the happy path" is actually sufficient, and the right thing to do. However, and bit overly simplified, but the non-happy-path is almost always the important parts of a system. It's the monitoring. Disaster recovery. Error handling and correction. Well thought out data models that prepare for future features, etc.
Most humans absolutely can write this with a suitable vector graphics tool such as inkscape or illustrator.
Surely, you're not suggesting that a fair comparison would be using a text editor?
If so, would you suggest an equivalent raster based task would only be fair, if the human would manually assigning RGB values to each pixel?
> Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.
Rule 2 follows rule 1.
Rule 3 & 4 is a variation of Keep it Simple, Stupid (KISS) from the 1960s.
... and... now I feel stupid, because I read the last part, which is summarizing it in the same way.
From my own observations, the types of people I previously observed to be sloppy in their thought processes and otherwise work, correlates almost perfectly with those that seem most eager to praise LLMs.
It's almost as if the ability to identify bullshit, makes you critical of the ultimate bullshit generator.
The remaining 99% had become a significant challenge to the greatest human achievement in distribution of knowledge.
If people used LLMs, knowing that all output is statistical garbage made to seem plausible (i.e. "hallusinations"), and that it just sometimes overlaps with reality, it would be a lot less dangerous.
There is not a single case of using LLMs that has lead to a news story, that isn't handily explained by conflating a BS-generator with Fact-machine.
Does this sound like I'm saying LLMs are bad? Well, in every single case where you need factual information, it's not only bad, it's dangerous and likely irresponsible.
But there are a lot of great uses when you don't need facts, or by simply knowing it isn't producing facts, makes it useful. In most of these cases, you know the facts yourself, and the LLM is making the draft, the mundane statistically inferable glue/structure. So, what are these cases?
- Directing attention in chaos: Suggest where focus needs attention from a human expert. (useful in a lot of areas, medicine, software development). - Media content: music, audio (fx, speech), 3d/2d art and assets and operations. - Text processing: drafting, contextual transformation, etc
Don't trust AI if the mushroom you picked is safe to eat. But use its 100% confident sounding answer for which mushroom it is, as a starting point to look up the information. Just make sure that the book about mushrooms was written before LLMs took off....
Answer is useful as is, and needS to be factually correct: Bad
Between [3] and [4] I added an assumption, without basis, of implied agency by [1].
War-mongering, and otherwise zero-sum mentality shouldn't make all sense if they have the technology to actually reach us. [3-body spoiler warning] Kinda like in the Three Body Problem. It was kinda silly how advanced the Trisolarian were, while still bothering traveling to earth, rather than approach the problem in any number of more obvious ways
But now that we're talking about these differences, I'd say that Apple users are notoriously complacent and defend Apple and their practices. So, perhaps in some part it is in an attempt to compensate for that? I'm still surprised how we've now accepted that Apple receives information pretty much every time we run a process (or rather, if it ran more than 12 hours ago, or has been changed).