Comment by noir_lord | Hacker News Reader

Comment by noir_lord | Hacker News Reader

Parent Full thread

It requires general intelligence and we don't even have a good understanding of how our's works or a particularly good way of quantifying it.

The counter argument is of course maybe you don't need to understand our kind of intelligence to create a different kind and that could well be true but then how do you determine if a system is intelligent.

Unless the new system is intelligent enough to reason with us on our level in a way we can "see" is intelligent it becomes a philosophical argument.

We also have a natural inclination towards anthropomorphising systems that mirror us, this is already a problem with LLM's and people overestimating their capabilities or forming actual emotional attachment.

Then there are those of us who know more about how they work who in theory should be more immune to that and aren't.

I added some stuff to my agent.md to make it sound less human and to communicate more like the machine be it is because I find the faked emotion extremely jarring.

It can't be sorry, it's a set of numbers, it sits in the linguistic uncanny valley.

Why do you think it requires general intelligence? The parent and I aren't being obtuse here: the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later. See Go, driving, art (you can complain about the quality of AI art, but it's winning competitions with human judges), etc.

A priori I'm not sure why you would think being a PM at a FAANG, deciding what color the login button should be, is any different.

Hence my saying.

We will never prove AI is intelligence.

We'll only prove humans are not.

I mean I could believe that there are some tasks which can't be done at acceptable performance until AI resembles something like Commander Data, I just don't think being a software PM is one of them.

I'm not entirely sure "general" intelligence is a meaningful concept but I take judgement to mean something like "the ability to make decisions under uncertainty without direct feedback." People seem to be able to do this using a lot of context and tacit knowledge. It's not really clear how you hill-climb to that capability through RL.

> the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later

True, but the history of AI research is also littered with AI researchers confidently predicting X job will be replaced by AI and being completely wrong because they don't actually understand what those jobs actually are. See Geoff Hinton predicting that Radiologists would be obsolete by ~2020, or predictions that truck drivers would all be replaced by self-driving tech.

I think AI wins against people when we start using KPIs in an industrial view of software like defects / LOC, decomposability, etc), or possibly even maintainability / understandability of a codebase. Gosh, even since the 1980s expert systems were already out-performing entire doctors' boards in diagnosing issues in patients so clearly technical performance in KPIs isn't the only measure by which we adopt technology in society.

When it comes to judgment calls for technical decisions, a lot of interesting innovations appear to come from rejecting conventions / averages in favor of a different set of constraints as a trade-off because we challenge the assumptions we make about the demands being asked of a solution / product. I'm thinking in the constellation of the apocryphal Steve Jobs quote about rejecting asking horse riders what they want because if we asked them they'd ask for a more reliable horse.

Reply on news.ycombinator.com