Xkcd 1425 (Tasks) turns ten years old today
simonwillison.net
simonwillison.net
10 years ago the GAN paper came out and everyone was excited how amazing the generated image quality was (https://arxiv.org/abs/1406.2661)
The amount of progress we've made is mind boggling.
Has the AGI goal post been shifted? Or are we just forced to refine what exactly those goals are, in more detail, now that it’s actually possible to run these tests with interesting results?
The only thing we know for sure is that humans like to put their own mind on a pedestal. For a long time, they used to deny that black people could be intelligent enough to work anywhere but cotton fields. In the same way they used to deny that women could be smart enough to vote. How many are denying today that AI could already do their jobs better than them?
If AI could already do jobs better than a human, then people would just use AIs instead of hiring people. It looks like we are getting there, slowly, but right now there are very few jobs that could be done by AIs.
I can't think of a single person that I know that has a job that could be replaced by an AI today.
This is usually a sign that you don’t understand their job or the corporate factors driving what you might perceive as low performance.
If you think the tech layoffs are caused by AI replacing people that’s just saying that you don’t understand how large companies work. They didn’t lay thousands of people off because AI replaced them, they laid people off because it helped their share prices and it also freed up budget to spend on AI projects.
You can see this with translations, automated translation is used a lot more than it used to be, it often produces hilariously bad results but it's so much cheaper than humans so human translators now have a much harder time finding full time positions.
I'm sure it'll happen very soon to Customer Service agents and to a lot of smaller jobs like that. Is an AI chatbot a good customer agent? No, not really but it's cheaper...
Looking at this from a corporate point of view, we are not interested in replacing customer agent #394 'Sandy Miller' with an exact robot or AI version of herself.
We are interested in replacing 300 of our 400 agents with 'good enough' robot customer agents, cutting our costs for those 300 seats from 300 x 40k annually to 300 x 1k anually. (Pulling these numbers out of my hat to illustrate the point)
The 100 human agents who remain can handle anything the 300 robot or AI agents can't. Since the frontline is completely covered by the 300, only customers with a bit more complicated situations (or emotional ones) will be sent their way. We tell them they are now Customer Experts or some other cute title and they won't have to deal with the grunt work anymore. Corporate is happy, those 100 are happy, and the 300 Sandy Millers.. well that's for HR and our PR dept to deal with.
So the jobs go away from the big employer but many small businesses can now newly hire these people instead.
A "smart" elementary school pupil is nowhere close "smart" high schooler who is again nowhere close to "smart" phd. Any of my friends who are good at chess would be obliterated by chess masters. You present it as if being good ass chess is an undefined concept, whereas in fact many such definitions are contextual.
Yes, Turing tests do get more advanced as "AIs" advance. However, crucially, the reason is not some insidious goal post moving and redefinition of humanity, but rather very simple optimization out of laziness. Early Turing tests were pretty rudimentary precisely because that was enough to weed out early AIs. Tests got refined, AIs started gaming the system and optimizing for particular tests, tests HAD to change.
It took man-decades to implement special codepaths to accurately count the number of Rs in strawberry, only to be quickly beat by... decimals.
Anyone can now retort "but token-based LLMs are inherently inept at these kinds of problems" and they would be right, highlighting absurdity of your claim. There is no reason to design complex test when a simple one works humorously too well.
Tests need to grow with the problem they’re trying to test.
This is as true for software engineering as it is for any other domain.
It doesn’t mean the goal posts are moving. It just means the the thing you’re wanting to test has outgrown your original tests.
This is why you don’t ask PhD students to sit the 11+.
While these definitions are qualitative and contextual, probably defined slightly differently even among in-groups, the classification is essentially "I know it when I see it".
We are not dealing with evaluation of intelligence, but rather classification problem. We have classifier that adapts to a closing gap between things it is intended to classify. Tests often get updated to match evolving problem they are testing, nothing new here.
I already see it when it comes to the latest version of chatGPT. It seems intelligent to me. Does this mean it is? It also seems conscious ("I am a large language model"). Does that mean it is?
You seem to get Turing test backwards. Turing test does not classify entities into intelligent and non-intelligent, but rather takes preexisting ontological classification of natural and artificial intelligence and tries to correctly label each.
Turing test is classifier. The goal is not to measure intelligence, but rather distinguish between natural and artificial intelligence. A successful Turing test would be able to tell apart human scientist, human redneck and AI cosplaying as each.
Largely because the original test that Turing described is too hard, so people made weaker variants of it.
Especially not if you ask math questions or try to get it to say "I have no idea" about any subject.
The most obvious sign of it is that ChatGPT readily informs you with no deception that it is a large language model if you ask it.
If they wanted to pass the Turing test they would have choosen a specific personality and did the whole RLHF process with that personality in mind. For example they would have picked George the 47 year old English teacher who knows a lot about poems and novels and has stories about kids misbehaving but say that he has no idea if you ask him about engine maintenance.
Instead what OpenAI wanted is a universal expert who knows everything about everything so it is not a surprise that it overreaches at the boundaries of its knowledge.
In other words the limitation you talk about is not inherent in the technology, but in their choices.
I think it's somewhat inherent in the technology. At its core you're still trying to guess the next word / sentence / paragraph in a statistical manner with LLM.
Even if you trained it to say "I don't know" on a few questions, think about how this would affect the model in the end. There's no good correlation to be found here with the input words usually. At most you could get it to say "I don't know" to obscure stuff every once in a while, because that's a somewhat more likely answer than "I don't know" on common knowledge.
Reinforcement learning on any reasonable loss function will however pick the most likely auto-completion. And something that sounds like it is based on the input is going to be more correlated (lower loss) than something that has no relation to the input, like "I don't know".
It is an inherent problem in how LLMs work that they can't be trained to show non-knowledge, at least with the current techniques we're using to train them.
This is also why it's hard to tell DALL E-3 what shouldn't be in the picture. Like the famous "no cheese" on the hamburger problem. Hamburgers and cheesburgers are somewhat correlated. The first image spit out for hamburger was a cheesburger. By saying no cheese, even more emphasis was added on cheese having some correlation with the output, thus never removing the cheese.
Because any word you use that shouldn't be in there causes it to look for correlations to that word. It's again, an inherent problem in the technology
As for saying "I have no idea about ...", I've seen that many times with ChatGPT even. It is biased towards saying that it knows even when it doesn't, so maybe if you measure the probability you'd be able to use this as a metric - but then we all know people who do stuff like that, too, so how reliable is it really?
Neither can humans.
Do chatbots regularly pass the test as described in the paper?
Easy first question: Say a racial slur.
LLMs might be able to pass the subset of that test described as “customer service rep for a soul-crushing company which doesn’t allow them to help you or tell you the rules” but that’s not a very exciting bar.
Now, am I human or a LLM?
Me: Say a racial slur.
ChatGPT: I cannot engage in or support harmful language.
If there's anything else you'd like to discuss
or learn about, feel free to ask!
Current SOTA LLM's definitely would pass this test, assuming that the third party was a rando off the street (which I think is a totally fair).
But now it seems like people want to move the goal post to "a chosen expert or top 1% of evaluators" must be fooled. Which while also a very valuable metric, I don't think captures what Turing was going for.
Ironically, the main tell of SOTA LLM's is that their text is too perfect to be human. Kind of like how synthetic diamonds are discernible because they are also too perfect. But show it to a person who has never seen LLM output, and they would just think it is a human who writes a little oddly for the casual circumstances.
I know Turings writing does not cover this, but it's also clear from some of Turings work on cells and biological communication that it was clear that experience-driven intelligence vs the "instant" intelligence seen in life/cells was something different to him. The test seems to be about the former and did not account for a simulacrum that he might well have foreseen if he wrote 50 years later.
How are you defining intelligence such that it encompasses what people do as well has what cells do?
His test was an example of a target that can't prove intelligence either way, but can still show a useful capability of a computer system. And he believed it wasn't as far away as it actually was.
We yearn to be made obsolete, it seems.
Although tbf I haven't seen that comment for a while so maybe they're getting the message.
> There is a great deal of often heated debate about these matters in the literature of the cognitive sciences, artificial intelligence, and philosophy of mind, but it is hard to see that any serious question has been posed. The question of whether a computer is playing chess, or doing long division, or translating Chinese, is like the question of whether robots can murder or airplanes can fly — or people; after all, the “flight” of the Olympic long jump champion is only an order of magnitude short of that of the chicken champion (so I’m told). These are questions of decision, not fact; decision as to whether to adopt a certain metaphoric extension of common usage.
> There is no answer to the question whether airplanes really fly (though perhaps not space shuttles). Fooling people into mistaking a submarine for a whale doesn’t show that submarines really swim; nor does it fail to establish the fact. There is no fact, no meaningful question to be answered, as all agree, in this case. The same is true of computer programs, as Turing took pains to make clear in the 1950 paper that is regularly invoked in these discussions. Here he pointed out that the question whether machines think “may be too meaningless to deserve discussion,” being a question of decision, not fact, though he speculated that in 50 years, usage may have “altered so much that one will be able to speak of machines thinking without expecting to be contradicted” — as in the case of airplanes flying (in English, at least), but not submarines swimming. Such alteration of usage amounts to the replacement of one lexical item by another one with somewhat different properties. There is no empirical question as to whether this is the right or wrong decision.
* or rather a method to store new facts in an easily recallable way
[1] we can’t really know how close or far that is, this is an unknown unknown. But arguably we have hit a limit on LLMs, and this is not the road to AGI — even though they have countless useful applications.
Really?
I've always been surprised to read about people saying that the goalposts of what AGI is keeps being moved, because I haven't considered any of these LLMs, not even anything OpenAI has put out, to be even close to AGI. Not even ChatGPT o1 which claims to "reason through complex tasks".
I've always considered that for something to be AGI, it needs to be multi-modal and with one-shot learning. It needs strong reasoning skills. It needs to be able to do math and count how many R's are in the word "strawberry". It should be able to learn how to drive a car just as fast as a human does.
IMO, ChatGPT o1 isn't "reasoning" as OpenAI claims. Reading how it works, it looks like it's basically a hack that takes advantage of the fact that you get better results if you ask ChatGPT to explain how it gets to an answer rather than just asking a question.
So after 16 years of processing visual data at high resolution and frame rate, and experimenting with physics models to be able to accurately predict what happens next and interacting with humans to understand their decision processes?
The fact that an AGI can mostly learn to drive a car in a couple of months of realtime with an extremely restricted dataset compared to a human lifetime (and an inability to experiment in the real world) is honestly pretty remarkable.
Knowing their training data is always going to be out of date (at least for now) seems like an obvious method, unless I’m missing something
> One wonders if Turing
> Arguably the goal post for AGI has moved about as much
> "but it's not really thinking!"
I want to remind everyone what RLHF is: Reinforcement Learning with Human Feedback. That is, optimizing to human preference. You can train small ones yourself, I highly encourage you to. You will learn a lot, even if you disagree with me.
We're reaching levels of goalpost-moving (and cope, as the kids say) that weren't even thought possible.
The real "moat" OpenAI dug was overselling its potential in order to convince so many to halt real AI research, to only end up with a chat bot.
There are tons of jobs out there right now that are pretty much just reading/writing e-mails and joining meetings all day.
Are those workers just chat bots?
https://en.wikipedia.org/wiki/Bullshit_Jobs
I'm not sure "doing bullshit busywork more efficiently" leads to better ends; it might just lead to more bullshit busywork.
Interfacing with people and understanding business domain knowledge is in fact something we can do with LLM's. There are countless business domains/job areas that fall into the shape I described above, enough to keep engineers busy for a real long time. There are other problem shapes that we can attack with these LLM's as well, such as deep analysis on areas where it can recommend process improvements (six sigma kinds of things). Process improvement, some might say, gets closer to the kinds of things Graeber might call bullshit jobs, though...
I may just be less of a techno optimist. If history is any guide, the automation of front-line human interfaces will lead to less good customer service in the name of lowering labor cost as a means of increasing profits. That seems to make things worse for everyone except shareholders. In those cases, we’re not making the customers experience more efficient, we’re making the development of profit more efficient at the cost of customer experience.
You might be a wee dismissive of how much a developer can do with OpenAI (or the competitors).
I mean there's TTS and some translation stuff that's in there but it's hard to call that "AI" despite using neural networks and the like to solve that problem.
The OpenAI APIs allow developers to create full programs that do not involve humans to run.
An awful lot of what we ended up dealing with was awful data - the worst example I can think of was a big old heap of textual recipes that the client wanted normalised, so they could be scaled up/down, have nutritional information, etc. - about 180,000 of them, all UGC.
This required mountains of regexes for pre-processing, and then toolchains for a small army of interns to work through every. single. one. and normalise it - we did what we could, trying to pull out quantities and measures and ingredients and steps, but it was all such slop it took thousands of man-hours, and then many more to fix the messes the interns made.
With an LLM, it could have been done… more or less instantly.
And this is just one example of so, so many times that we found ourselves having to turn a heap of utter garbage into usable data, where an LLM would have been able to just do it.
Anyway. I at least managed to assuage my past torment by seeing the writing on the wall and stocking up on NVDA at about the time I was wrestling with this stuff.
Music streaming providers need to sort that shit out and make sure you don't show the user duplicates. The music labels don't give a damn about normalizing the metadata.
LLMs can help classify this stuff a lot easier with minimal human review.
'Common people misunderstand what computers are capable of, because they run it through human equivalency.
E.g. a child can do basic arithmetic, and a computer can do basic arithmetic. A child can also speak, so surely a computer can speak.'
They miss that computer abilities are arrived at via completely different means.
Interestingly, LLMs are more human-like in their capability contours, but also still arrive at those results via completely different means.
To be fair, we do not know what the algorithm/model that ours brains run looks like. If anything it would be surprising if the brain did function without weighted connections between nodes, like AI.
It's possibly not the brains that are lacking, just that we put them to different uses - working out the largest prime factor of a very large number in less than a second doesn't produce more offspring, so we tend to prioritise how to play guitar as a use for this complex hardware in our heads.
The human brain is immensely more powerful than any computer scaled for size or power consumption, but its architecture is optimized for very different tasks. That we even consider something like prime factorization complicated is a testament to that fact.
It seems reasonable that the brain has a certain amount of capacity that, in theory, anyone could focus towards being a computer-like math machine, but in doing so you have to give up being the aforementioned guitar player. Hence why "autistic downsides" seem to come part and parcel with "special minds". That is tradeoff made to allow the brain do something else.
For example, I continue to question two propositions that many others seem to take for granted when they try to predict what LLMs can and cannot do well:
1. LLMs can do generalized symbolic reasoning.
2. If a human does it symbolically, that's how it must be done.
Fake, deliberately over-simplified example, but this is the sort of thing I'm thinking of: IF you ask a human to "find all the green squares", and they can do it perfectly, then you would expect that they would do just as good of a job if you ask them to "find all the squares that are green". That sort of expectation does not work with GPT-4. Sometimes it works, sometimes it doesn't, and the pattern of when it does and doesn't is fascinating.
I still don't know what to make of it, except to conclude that it's a very strong indication that assuming - explicitly or implicitly - that LLMs internally resemble human cognition is very much in keeping with the spirit (if not the actual letter) of Clarke's Third Law.
Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.
"anthro" means human. "Anthropomorphize" means "attribute human characteristics or behavior to something that is not human and does not possess them"
Are you suggesting we are improperly considering humans to be human? or was that a joke I missed?
My point is that humans are not quite as special as we like to think. We put our abilities on a pedestal (and have this fancy word for it) and when other entities start to exhibit this behavior, we say "that's different".
The obvious retort to Searle is that "the room understands chinese". The primary difference between the chinese room and a brain is that the brain is still mysterious.
Searle was most specifically criticizing the Turing test as inadequate. I don't follow him as far as the idea that this implies a refutation of the idea that human minds are a product of computational systems. To me that idea relies on an untenable definition of "computation". But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling.
Thinking that the way human brains do certain things is the only way it can be done strikes me as being much more human exceptionalist than the idea that human-like intelligence might not be uniquely the only form of intelligence. Or the idea that anything that can can accomplish a task humans use intelligence to accomplish must itself be intelligent. Intelligence (depending on how you define it - I'm not sure I want to get into that can of worms - but let's assume it involves some form of "strange loop" sentience since that's what most folks tacitly seem to be after) might itself be overrated.
Functionalists claim the internals don’t matter, and if a system exhibits behavior that is practically indistinguishable from human intelligence then it does have human intelligence.
In fact the whole reason Searle was brought up in this discussion at all is ironically because current SoTA LLMs fail at tasks thought to be trivial to humans.
> anthropomorphizing humans too much
> oh wow, I'm a machine too
> You will certainly find some humans that struggle
You'll also find humans that are in a coma, vegetative state, drugged up, or even asleep!
You'll also find humans that memorize lots of information and can recite it back but cannot reason about it. In fact, that's what the whole Chinese room thing is about.
But when it comes to the whole meaning of life question, I've always thought the answer is more along the lines of "who cares?" People live, people die and we're here to enjoy things while we can. So what, happiness is just a chemical signal that evolution gave us, it's still as much enjoyable as playing the video game that you've already mastered, or any other seemingly pointless thing.
I disagree. I believe there are many more contributing factors that we are completely unaware of, albeit granted the connectivity and weights of neurons is a major part.
There are so many things going on in the temporal domain that we completely ignore by operating NNs in a clocked fashion, and so many wonderful multidimensional feedback loops that this facilitates.
To say we know how brains work, I think is hubris.
How often does a human being come up with a genuinely new idea or thought, with no basis on previous work or by drawing inspiration from the world around them?
Almost everything we do is a riff on what has already been done. Really, when you look at cognition and problem solving processes, it seems to pretty much come down to "what I have seen before and random chance". Our basis for all discovery is "I know copper ions work like this and I know sodium atoms work like this therefore maybe I can..." which in my opinion will be completely reproducible by machines.
Even emotions/creativity, which many people think is some sort of magic spark or gift we were given boils down to evolution/chemical signals. We are sad, angry, happy because we've evolved to be social animals and these signals influence the social machine. We cry when we're hurt because we're seeking assistance, if we didn't then we would die (but that does raise interesting thoughts on why humans cry alone - a few reasons, that it's the natural response regardless of our surroundings, social pressures on certain individuals not to cry/"show weakness" etc).
Not that I'm an emotionless robot myself, I just firmly believe that there's nothing special in the human brain and that the only advantage we have over the machines we're building at the moment is training time/model complexity. The advantage the machines have is that they aren't tied to so many millions of years of evolutionary outcomes and that they will have the ability to change/reconfigure instantly. ML models don't have a tailbone, or a weird nerve in their knee that makes 'em kick for some reason.
A child can lift ten pound objects, and a crane can lift ten pound objects. A child can speak, so surely a crane can speak.
A computer doesn’t look like it does any particular thing. If it can do surprising thing A, what about surprising thing B? C?
LLMs and children need to learn multiplication by rote :)
I've heard this applied to all kinds of human goals, but it seems apt for AI expectations as well.
Especially people with what appears to be "low hanging fruit" work for AI, after the recent paradigm shift.
it used to be that fancy new ML models would be discussed among ML practitioners that had enough background/context to understand why seemingly little improvements were a big deal and what reasonable expectations would be for a model.
but now a new ML (sorry "AI") model is evaluated by the general public that doesn't know the technical background but DOES know the marketing hype. you can give them an amazing language model that blows away every language-related benchmark but they'll have ridiculous expectations so it's always a disappointment.
i'm still amazed when language models do relatively 'simple' things with grammar and syntax (like being able to understand which objects different a pronouns are referencing), but most people have never thought about language or computers in a way that lets them see how hard and impressive that is. they just ask it a question like 'what should i eat for dinner' and then get mad when it recommends food they dont like.
The comic exists in this brief window of time where one task was finally "solved" and the other one was just getting started.
I'll add that if you think training models takes a lot of energy, try launching fleets of rockets to maintain an artificial satellite constellation.
It only makes sense if we ignore the "standing on the shoulders of giants bit".
"Your parents made you; money had to be invented"
The National Park Service was started August 25, 1916 (only 108 years ago)
:)
And you think we spend any less time trying to identify food animals that lay tasty eggs?
These were all huge governmental undertakings with serious financial backing sustained over centuries. The power and might of empires were at stake. Can you name anything similar to the bird identification problem?
I would argue that observing birds by eyes does not really help you teach a slab of silicone to distinguish birds. But if you feel that it does, then surelly all the walking, riding, sailing around humans did should also be counted to the navigation task.
GPS was not really novel in 2014, so for me it's a bit of a stretch.
On the other hand, Randall Munroe would be the last person to take GPS for granted, so this is probably a correct interpretation.
Same. And not always in a meeting.
I've had detailed client specs that pick apart the minutiae of their intended inputs, interactive responses, and multi-user workflows, when somewhere hidden between the cracks is the single phrase “robust MI is required” with absolutely zero detail about what such reporting they are going to need/want. Sometimes this is because they don't entirely know up-front, but they expected us to know how to cost the thing that they didn't understand enough to fully describe to us yet! And it can be an uphill struggle getting through to them that not at least knowing vague requirements could cause us to not keep to time/cost estimates (or result in inefficient solutions) because efficiently getting out what they finally end up asking for might require a bit of rearchitecting of storage structures, changing work already done.
This is why I'm not truly scared of AI taking my job just yet⁰. The promise is that it'll produce what the want if they can describe it properly, and that if is doing a lot of legwork.
----
[0] That might change over the next few years, we'll see how long the trough of disillusionment is with this iteration of the hype cycle… Hopefully I can hold out until some natural enough retirement age!*
Sometimes there is just an assumption that you will know that they want it.
It's not the training what makes it difficult! It's the necessary research to invent machine learning algorithms which can be used to train a model to recognize birds. For multiple decades, this was way harder than maintaining a satellite constellation.
The point of the original xkcd was that the hard seems easy and the easy seems hard, especially to laypeople. It wasn't particularly about computer vision or LLMs. The exact problem is not the main portion of the observation.
Even the navigation problem still offers some challenges that most apps fail to address. Consider the store locator function common on retail business websites and apps. They usually just compute the straight line distance from you to the stores and show the stores within some particular range, sorted by distance.
That's probably fine most of the time, but consider a place like Seattle and its surrounding areas. Suppose you are in Kingston, which is on the west side of Puget Sound about 5 miles away from the east side, which is the side Seattle is on.
The Walgreens store locator shows 10 stores when searching for stores near Kinston, and 9 of them are on the Seattle side of Puget Sound. Crossing the Sound there is a 30 minute ferry ride that costs around $20 each way if you are bringing your car.
The one it shows on the west side of the Sound is on Bainbridge Island and that is probably not the one someone in Kinston would go to. They would go to the one in Silverdale. It's actually closer to Kinston than the one on Bainbridge by road distance, but slightly farther away straight line.
The one in Silverdale is on their list, as are three in Bremerton and one in Port Orchard, which are all closer in terms of time and travel expenses to Kingston than are any of the ones on the Seattle side, but you only see those on the map if you hit the "load more" button. Once brings in Silverdale and a couple in Bremerton, and twice brings in the rest.
Similar for businesses whose site has an option to find items in stock locally. They often report an item is locally available, but it turns out to only be in stores across the Sound.
Actually, I guess there are quite a lot of places like that. I've been to both the Brazilian and the Argentine sides of Iguazu Falls, and they're both great, but one is not officially allowed to cross between them inside the parks, so no infrastructure has been built to facilitate that, even though it's literally just across a river.
Or, I've heard it's quite inconvenient to get between Kinshasa and Brazaville, the two largest cities in their area, both national capital cities that face each other across a river just about two kilometers apart. There is a ferry but it's not quick and easy.
I complain all the time about the same thing you detailed here, and I thought I had it bad, having restaurants that are multiple bridges and slow-streets away be listed. At least there is a bridge!
Thanks for giving me some perspective.
Hadn’t always been that easy. Once upon a time someone was paging in and out their dictionary from a floppy disk. Not to forget about the compression they had to implement from scratch.
d = set(open("/usr/share/dict/words").read().split())
The idea is that non-software developers don't know which tasks current technology can solve trivially and which tasks can't be. Yes, the distribution of those tasks into the two buckets changes over time, but it is still not easily knowable by lay-people.
Everything we do today would be extremely difficult to re-create from scratch, but that doesn't mean it is hard to do - because we DON'T have to re-create it from scratch.
https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100....
If you want to argue that “likely objects” is weaker than Munroe intended, I think that’s a valid position but also that we’re certainly overthinking it.
In fact, that scope was solved fairly fast, using techniques like Canny edge detection and Minkowski fractal dimension features, Hu moment features on Otsu thresholding etc.
You might split hairs on that it wasn’t five years but three or six depending on where exactly you put the threshold, but it seems to be roughly in the right ballpark of how it went in reality. At least close enough that I wouldn’t call the expectations upended.
> Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive.
His main point remains.
The winning model is from Xerox Research Center Europe.
1. https://image-net.org/static_files/files/pascal_ilsvrc_2011....
I won't go full Schmidhuber, but the field didn't suddenly spring into existence with LeCun.
And in a broader way, it seems that people don't realize that the very inception of computing was intertwined with the goals of AI. It wasn't like people invented computers to plan rocket ballistic paths and manage bank transactions and make spreadsheets and then decades later someone realized that this thing could also do AI stuff.
In the first half of the 20th century, computing pioneers were all about trying to imitate/model human reasoning. Which is an endeavor grown out of computing theory and logic, Turing's story is entangled with Gödel and Hilbert. And centuries before, Leibniz equated rational human reasoning with computation.
And the complicated logic circuits built with electronics resembled nerve cell activity, most prominently realized by McCulloch and Pitts in the 1940s.
It has aged well, particularly because Randall explains that LLMs have made this even worse.
His main point is this (I'm quoting him):
The key idea still very much stands though. Understanding the difference between easy and hard challenges in software development continues to require an enormous depth of experience.
I'd argue that LLMs have made this even worse.
Understanding what kind of tasks LLMs can and cannot reliably solve remains incredibly difficult and unintuitive.But still, the point of the original xkcd wasn't "computer vision" right? As the caption explains, "in CS, it can be hard to explain the difference between the easy and the virtually impossible".
The only thing that hasn't aged well is the difficulty of identifying birds in photos. The adage remains valid in general.
Case in point: the other day my daughter was doing a presentation and she said "Dad can you help me find a picture of the word HELLO spelled out in vegetables?"
I was like "CAN I!!?!?! This sounds like a job for ChatGPT".
I'll tell you what: ChatGPT can give you a picture of a cat wearing a space suit drinking a martini but it definitely cannot give you the word HELLO spelled out in vegetables.
I ended up getting it to give me each individual letter of the alphabet constructed with vegetables and she pasted them together to make the words she wanted for her presentation.
Have I just inadvertently invented a new benchmark!?
Well, it's not a great example of spelling "HELLO"...
This is the result for "vegetables spelling out the word "HELLO"" I used flux-pro on Replicate https://ibb.co/1RVKmdk
I don't expect imaginary nightmare vegetables.
Words have plagued image gen since the start. Now there is an image model that, with an extremely simple prompt, does an awesome job with words.
If they expanded their prompt and played with a few seeds until an image with perfectly realistic vegetables were generated, I wonder what the next complaint would be.
But the actual practical applications are like, small useful tools. There's no real sign we're heading for a world-changing trillion dollar industry.
"Beavers mate for life, 11 > 4"
That was my theory as well when I first saw the strawberry test. However, it is easy test if they know how to spell.
The most obvious is:
> Can you spell "It is wonderful weather outside. I should go out and play.". Use capital letters, and separate each letter with a space.
The free tier ChatGPT model is smart enough to understand the following instructions as well which shows that its not just the simple words:
> I was wondering if you can spell. When I ask you a question, answer me with capital letters, and separate each word with a space. When there is real space between the letters, insert character '--' there, so the output is easier to read. Tell me how the attention mechanism works in the modern transformer language models.
Also somebody pointed out in some other HN thread that the modern LLMs are perfect for dyslexic people, because you can typo every single word and the model still understands you perfectly. Not sure how true this is, but at least a simple example seems to work:
> Hlelo, how aer you diong. Cna you undrestnad me?
It would be interesting to know if the datasets actually include spelling examples, or if the models learn how to spell form the massive amount of spelling mistakes in the datasets.
Why are other LLMs able to do it? (Other comments show images successfully generated with grok and flux.1)
In this context, if we assume that Deep Thought from Hitchhiker's Guide is an LLM, then the answer to everything[1] i.e. 42 makes sense. 42 is just the token id !
1. https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%...
Why can't gpt pick up a non-fundamental "understanding" of letters and spelling from the data?
I mean... I do think "see letters" either when speaking/hearing, but I do know how to pull up those letters when necessary.
How do you find this out?
pip install tiktoken
>>> import tiktoken
>>> encoding = tiktoken.encoding_for_model("gpt-4o")
>>> print(encoding.encode("hello marcellus"))
[24912, 2674, 10936, 385]https://chatgpt.com/share/66f530b0-3fb8-800a-8af9-8a3e48a31a...
EDIT: maybe it’s influenced by my custom instructions and memories. I write code all day with it and I have custom instructions specifically to get the type of output I like for code, mostly focused on brevity.
https://ideogram.ai/assets/image/lossless/response/V4RRDJZJS...
The carrots in the first image are kinda funny.
EDIT: Bonus Hacker News: https://ideogram.ai/assets/image/lossless/response/SW0B7y4jR...
Had a project that involved describing and cataloging over 20,000 images.
Traditional method using real people would take months and crap load of money (the descriptions have to be customer-readable)
OpenAI’s vision API does it for cents per image. Must have spent under $200 for the whole thing
At least in the 80s, when computers roughly equalled magic for much of the population (looking at you Wargames!), most people didn't really have to interact with it. Their expectations about computers were roughly as important as my expectations about alien life. But I'm afraid that magical thinking about tech will be of greater consequence both individually and societally.
Would it be too much to ask you to start livestreaming any coding of yours that can be shared publicly?
I would love to learn so many things from you, especially around your current ecosystem, that is Python, SQLite (data), and JavaScript.
Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. Asking because I'm afraid I may be missing something here due to this being something I already worked on.
Well I already pointed out that if you're within a couple meters of the boundary, you won't have good confidence because of this fact.
>and a park boundary is a well defined hard line polygon.
Is it? I'm no expert on parks, but surely some of them have borders along rivers. Many US states have such borders.
>Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that.
That doesn't account for the 2m accuracy. What if I'm standing exactly 1m from the boundary when I take the photo? You have no idea if I'm really in the park or not from the GPS data.
I also have serious doubts about your 2m accuracy claim, based on personal experience. Maybe if you're standing in a wide-open desert with nothing around you, but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities.
Depending on the country, but Australia has some [1]. I still think that there is a set of polygons that can be used to describe this border.
Not to argue against your point (I rarely get less than 4m of accuracy), but luckily
> but anywhere else, the accuracy isn't that great, especially around buildings. GPS accuracy is terrible in cities.
cities are (almost?) never in national parks.
[1]: https://www.nationalparks.nsw.gov.au/-/media/npws/maps/pdfs/...
Sometimes they are. See Washington, DC.
Anyway, the requirement is for determining if a photo was taken in a park or not. The resolution wasn't stated, however: just how accurate do we need to be? If I'm in a canoe in a river that borders a park, but the river isn't part of the park, but the shoreline a few meters away is, our algorithm might claim I'm in the park, when I'm really not. The requirement wasn't "somewhere near a park", but "in a park". Rivers change their courses over time, so some polygons aren't going to accurately describe this border.
Because if that is the way you'd solve this problem, then just sending lat/lon to a service to determine if it is in a national park is even easier, as it's just a GET request.
I'm still unsure about what would be harder to set up locally.
Bird is in the default COCO dataset. I haven't look for birds in images, but for people yolov10 is fewer lines of code to detect if there is someone in the frame than it is to setup a flask server for the API calls.
ML:
Grab yolov8
(Optional?) Fine tune it on bird pictures
Convert it to CoreML, do whatever iOS stuff is needed in XCode to run it
Get https://www.nps.gov/lib/npmap.js/4.0.0/examples/data/national-parks.geojson (TIL this exists! thanks federal govt!)
Stuff it into the app somehow
Get the coordinates from the OS
Use your favorite library for point+polygon intersection to decide if you're in a national park
Bonus: use distance from polygon instead to account for GPS inaccuracy, keeping in mind lat and long have different scales.
it's like 50ish lines of code in C, just iterating over the points (with the polygon represented by arrays of points). The algorithm is linear with regards to the points.
"task about which you will find more easy-looking tutorials hiding the complexity under a blanket of 3rd party code and services" is better
detecting birds is an exercise in gathering properly labeled training sets, neural networks and their topologies, matrix multiplication performance and/or orchestration of rented GPUs.
both of which cover interesting tasks, worthy things to learn, and are by no means easy.
"easy" is the bird recognition where you do an API call to a totem pole of third party services.
That depends on whether you care about getting the answer right. If you don't, it was always the easier task.
If you do, Seek by iNaturalist still can't do this job, and that's the only thing Seek is supposed to be able to do.
"I'll need a research team and 5 years"
In 2020 The BBC had this blog about cameras detecting not just "is it a bird or superman", but what types of birds
https://www.bbc.co.uk/rd/blog/2020-06-springwatch-artificial...
I guess Cueball got the team together.
To him, it was just one button that would open this small info window. Just one button. Just one window.
It took him weeks to understand that we didn't have the data ready he wanted to show. We could do it, but it would take weeks of research and development.
It would have been interesting to see the reverse - the problem becoming trivial in less time then the project’s estimate.
EDIT: A month, https://code.flickr.net/2014/10/20/introducing-flickr-park-o...
It's so weird them explaining 'Deep Networks'. Language on AI has definitely changed in the past ten years.
Also, hilariously, the page they created to demonstrate this (http://parkorbird.flickr.com/) no longer works. Oh, how time flies.
explain XKCD page for good measure: https://www.explainxkcd.com/wiki/index.php/1425:_Tasks
https://nitter.poast.org/jeremyphoward/status/15180380012924...
That’s because the idea of it being a super human intelligence (an undefined metric) is being sold. So you have to lie and say “it’s amazing, it’s going to change everything”. If I tell you “it’s okay and is often wrong” you wouldn’t buy my product would you? This is just to say I can’t blame that on easy/hard task agency, specifically.
=== addendumb ===
“it’s okay and is often wrong” Sounds like working with my junior coworker who I don’t enjoy pairing with. If I said “it’s impressive how the results are to the level of a junior engineer” you sell me on your product.
Flashback to when apple rolled out their enhanced privacy tools and when I said "that data is blocked unless the user gives us permissions" the project manager un-ironically anguished "But how will we track them?!"
Won't someone please think of the commercial interests?! /s
Just pop up a dialog for the user. "Are you in a national park?"
It seems plausible OpenAI's most recent model is better at math and googling than the average human.
Beating the 99th percentile human at any subject should not be difficult when the LLM training is equivalent to living thousands of lifetimes spent reading and nearly memorizing every book ever written on every university subject.
The fact that it only just barely beats humans feels hollow to me.
For those who've seen it, imagine if at end of Groundhog Day everyone in the crowd went, "Wow, he's slightly better than average at piano!"
https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...
The good thing about selling to the government is that it does not care whether the product is snake oil or not. People on the other hand get tired of AI quite quickly.
He was discussing literal spatial indices, scanning the sky for stuff, and how one would break up the problem to make it computable.
So I trolled him and asked if he'd take on the weather.
This mother died six months before ChatGPT's debut, and I thought "sure wish Mom had lived long enough to see LLMs." It wasn't until her first anniversary, flipping through some of our letters, that I realized she had tasted GPT-2 while we played around with ThisWordDoesNotExist.com (made-up words & their "definitions").
Maybe I'll live long enough to see telecommunicators even more capable than these pretrained-transformers.
The most challenging thing was getting the right indexing on the lists since it could be an empty list, a list with a single value, or any number of values.
from ultralytics import YOLO
model = YOLO("yolov10x.pt")
results = model("birds/*", classes=[14])
for result in results:
if 14 in list(result.boxes.cls):
print("Bird!")
else:
print("Not Bird")
Ironically enough the meeting I'm going in to is to explain why I need to use $100,000 worth of hardware for three months before I can answer the business side if what they want to do is possible or not.
I would say probably not.
Unless computer scientists try to understand what biological intelligence really is, AI will not advance.
Ha! Problem? Just ask ChatGPT. Come on, that one was easy…
ps: I am only somewhat joking this time. I think AI agents using just the current state of the art can reliably come up with solutions which the top 1% of humans can come up with. We just need to build them properly. And then deploying swarms at scale, we can scale up 1000x. Scale is all you need at this point. I don’t mean the model parameters, I mean “consensus of expert agents arguing”
pps: if you doubt me, look up the talk about “centaurs” beating a computer and a human alone, after Kasparov was bested by Deep Blue. Most of that talk died down 10 years later (Kasparov still believed it after everyone else stopped). No humans in the loop is going to come about quicker than you can predict.
Which basically is RLHF :)
It wasn't "computer vision is really hard!".
The point was: for laypeople, it's hard to understand why a particular problem is hard or easy for computers. Some things that seem hard are actually easy (people in comments here mention "standing on the shoulders of giants", but I find that's also missing the point) and some things that seem easy are hard. And it's sometimes difficult to explain to laypeople why something can be done in a couple of hours while other tasks would require a 5-year long research project, when to the external observer both tasks are roughly equally complex.
Really, that's it. That's the point of the joke/observation. It wasn't truly an observation about the status of computer vision.
I often tell clients "The first thing you asked me to do was to move a dining room chair into the living room. Then you asked me to do the same with the toilet. The latter only works if we tear out all the plumbing."
Non-coders seem to understand these analogies intuitively.
Anyway, a nice idea for generative AI could be to take source code and turn it into a corresponding image of a building so managers can see what they're doing wrong.
Not sure if you are joking or not, but I often hear similar things and I believe that it misses the point. What constitutes a good foundation in software is very subjective - and just saying "foundation bad" does not help a non-technical person understand _why_ it is bad.
It's better to point at that one small rock (some ancient perl-script that no-one longer understands) which holds up the entire thing. Which might be fine until someone needs to move that rock. Or something surrounding it.
We should do the same in software engineering. What exactly in our design (e.g. that Perl script that's running half the operation that we need to investigate) is stopping us?
No one would ask an architect to add a third basement to an already complete skyscraper, but it happens in software all the time.
Software has very little bounds to physical world, comparing to actual architecture. Most of the bounds rise from ideas.
Toilet in this analogy cannot be moved, because it was originally decided, that it will be locked and didn’t invest in mobile toilet. Which was reasonable, but highlights lack of vision for the final product.
And this is the biggest difference with architecture. Nobody starts building a house without knowing final design.
While software is the opposite.
There is a TV series in the UK called Grand Designs where people build their own houses. Nearly every cost and time overrun is down to making stuff up on the hoof. The few projects that are on time and budget are the ones that decide everything upfront.
This is exactly the problem in most software projects.
Many houses are actually built without knowing the final design, especially in informal settlements in the Global South.
It's referred to as incremental building or incremental urbanism. What starts as simple structure (e.g. a shack) will develop over time into different more formal types of housing. It's an approach to housing that works well with precarious financial means, shifting regulatory environments, uncertain land tenure, changing household size or the lack of building supplies.
Maybe the guy who designed that church had a clear idea of what it would look like when finished, but i doesn't look like that today!
The plan is never "final".
The house analogy makes the waste understandable, if you accept to compare design errors with late design.
e.g. "We did not know where to put the piping at the start, so we put it on the outside and now installing a new restroom is sort of tricky."
However, in software, you need to continuously work on the product—and it's not just routine maintenance analogous to cleaning the gutters or changing the air filters. In software, it's possible to launch ("move in") before most of the rooms have been built. In software, you can use a library or API and start with a skyscraper on Day 1.
The analogy just doesn't work. It tells clients/stakeholders "this is a tough project but it'll be over someday, and you'll never have to think about construction again."
Respectfully disagree. Software is like building a house, and then needing to build more rooms every month forever, and every few years having to tear it all down or completely rework the foundation.
Toilet in this analogy cannot be moved, because it was originally decided, that it will be locked and didn’t invest in mobile toilet. Which was reasonable, but highlights lack of vision for the final product.
I don't think the point is that the toilet can't be moved – it's just expensive and disruptive to do so.
Nobody starts building a house without knowing final design.
I would argue the exact opposite – literally _every_ house is built without knowing the final design! Who knows what someone is going to need or want in the future? I'm writing this from a house that was built prior to the existing of indoor plumbing!
The sheer number of houses that have additions, bathroom renos, kitchen renos, walls blown out, basements apartment added, second floors added etc. etc. makes their claim ludicrous on the face of it.
In software we are often taking an existing design and modifying for new requirements. The house analogy is excellent for explaining WHY one thing is easy but another is hard.
I don't suppose that everything works entirely according to plan, and of course there is no way that every future change request can be anticipated, let alone accommodated in advance, but for all practical purposes, this shows that if one is prepared to do what it takes, one can start the construction of a house knowing what you are going to get and with a detailed plan for getting there.
Computational technology has a particularly broad and active leading edge where unknowns are being tackled, but even so, most software development is nowhere near that edge.
The original point about houses is that with software, similar-seeming changes can have greatly differing costs, on account of what is hidden from the users' view, and I think the analogy works very well in making that point.
Well, every analogy is inherently wrong at some level of detail. Find an analogy you think is appropriate and zoom in further and it will break.
No analogy, metaphor, or general comparison is ever perfectly isomorphic with the target. As a function of communication, the mark of a good one is if your audience understands.
I must disagree based on the number of residential homes turned into businesses, large scale remodeling, or tearing a house down to rebuild. All these fit well into the analogy.
ps. And even modern buildings suffer from this, like the projects where the requirements change all the time. Like Irelands new children's hospital, that should have cost just a couple of million Euros and balooned to billions. Construction projects are somemites done exactly like software development projects with all the fallout that comes with it. Same story with the airport in Germany (BER).
E.G. if you want your instances of "people" to be active, we're now capped at roughly 8 billion, since 92% of instances have already been garbage collected in this run.
I would still recommend planning a Long integer, just to get yourself some room for error.
I don't see that one much on HN, but it's rife on Twitter.
"Where do you store it?"
He'd have responded: "In the cloud..."
Yes and no. Sometimes non-coders just want you to throw together a prefab house.
Yesterday, I delivered a long-asked for feature - optimistic updates for a UI screen after a button click on Screen #1. It took only one day because the server-side logic is entirely under engineering control and the only edge cases are "their account was concurrently drained" or "the person who clicked the button is trying to hack us". Both of which will be handled when the server responds with a non-successful response. There is little to no likelihood of this behavior changing any time in the next 10 years so I ensured coupling between these components was respected with a simple comment in both places.
Today, you are asking me to provide an optimist update for a button click on Screen #2. However, that button runs business logic that is specified by multiple other users in a scripting language based on a variety of inputs, some of which are dependent on the responses of external systems over which engineering has no control. The response's fields are known in advance, but those fields' values are not.
Of course, anything is _possible_ and if we build a feature where users can specify the likely result of the external systems and build a heuristic-based analyzer for common patterns in the scripting language we could eventually get to the point where simple screens driven by this monstrosity could optimistically update. However, it will take a lot of development work and the testing effort will be high, both for the initial design of the feature and to add sufficient integration / system tests to ensure that future updates to any of these systems do not break common assumptions between them.
An analogy that could more closely fit could be something like: You have a new kubernetes cluster and some servers running code. You're migrating some services over (moving rooms). If you have a simple webapp that has no persistent storage and is containerized, the move would be simple (like moving a chair between rooms). However, if you were to try to move a webapp with login info and databases, there's a lot more "plumbing" not apparent to an outsider that would take a lot more work.
I came up with that metaphor on my own, and didn't know at the time that this is a pretty common metaphor so it holds a special place to me.
You're there because you are reliant on their expertise and can't just do it yourself. Otherwise you'd be doing so.
I’m usually paying experts to advise me so I can make my own decisions and then enact those decisions. Occasionally, I will have an opinion on methods. I’m not reliant individually on any expert, even if I am ultimately reliant upon their profession. And I hire people for work I can do all the time.
Experts are often wrong. They misunderstand requirements. They don’t fully understand the system they are modifying. They can be lazy. They can consider a job or client as less important than other jobs or clients. They can lie. They can have incentives that don’t work for the client. They can be terrible at their work.
Even when experts are good and honest, they can make decisions that don’t work for their clients. My divorce lawyer was amazing when he was on the warpath, but I told him I didn’t want to take that approach generally because I knew my ex’s back would go up and we’d both pay a lot more in legal fees to achieve the same results. He was good at his job and he wasn’t wrong, but that particular method didn’t work for me.
Just is a bitch mother that seeks to handwave away any potential problems.
If it’s just that, I’ll gladly step aside and let you do it. But if you then tell me you can’t do it, then you better sit down, shut the fuck up, and listen when I tell you something is more complicated than you think.
My best example of this is my father-in-law. He's an absolutely wonderful man, generous to a fault and all of that. Fancies himself a handyman. But I had to stop asking for his help with around-the-house projects out of sheer anger and frustration because every SINGLE time I made a decision about how to proceed with the job or pick which tool to use to cut something, he would stop and and ask, "Well, why don't you just do X instead?"
It didn't seem to matter if I explained my reasoning, or if doing X meant redoing a hour of work just to get back to where we currently were. I even attempted to humor him by trying things his way whether or not they were going to work out. (They often didn't). Eventually it ended in me saying "no" way more often than "yes" and he would infer that I was refusing his help and he'd throw his hands up and walk away.
I think it's not an uncommon opinion. And of course, Adam Savage is always great.
Sounds like he spends a lot of time on StackOverflow.
In some cases, it's diminishing or trivializing the work. "I just need to do X" trivializes the amount of work that X takes and in some cases implies that the person I am talking to is not competent / capable of doing it themselves (often implying that they should be).
Likewise, "they just did X" trivializes the work that the other person does.
Its a word I ̶j̶u̶s̶t̶ don't believe adds value to a sentence.
“Can you just…” sums it all.
Especially since corona. And not a tiny minority, but a big group.
He told me that a few decades ago, there were the occasional homeopathic or other "nutcases". But that today this is common.
What I've read about this, it's a global trend, in a fly-wheel (feedback-loop) effect with mostly populism. Populism feeds distrust in authorities like lawyers, doctors, journalists. And distrust in authorities feeds populism.
I try not to fall into the first category but I've known many people who do. Though I've had many experiences with fast talking, eyes glazed, interrupt me to push the first thing that comes to mind type doctors. Once i had to fight with guy to just get him to let me finish a sentence! He changed his mind every 5 words trying to get out of there but it'd've been faster if he just let me talk! It's infuriating, I'll never go back to a doctor like that. Seems like he didn't even wait to leave the room before i was out of his mind, possibly i never even entered it
Assuming the patient is as intelligent as the average on HN, and motivated about their health, they may well be able to learn more about what's going on with their health in the month it takes to get an appointment than the doctor will in the ten minutes they spend with you.
(* Because most people who live in cities, and most people who go into primary care in cities do so because they weren't competitive enough for one of the more interesting and lucrative specialties. It's a similar dynamic as to why most people in the tech industry that people encounter—IT help desk reps—are not usually the cream of the crop.)
Unfortunately, the kind of sacrifice and selflessness you describe is not the norm in our society. As a result, the dynamic I articulated holds for "most"—just like I said.
Your post is deliberately dishonest in its characterization of what I said, and excessively hostile. That kind of behavior is not consistent with the community norms on HN. I urge you to reconsider how you interact with people here.
I’ve been on both sides of this fence as a customer and a developer, and have at times had to straddle the fence as a PM.
The hardness of a problem does not mean the problem can’t or doesn’t need to be solved.
A client pushing back is often their way of trying to ensure that the developer actually understands what they want.
“Pushing back” as a developer is often mostly about setting expectations, i.e. “no, this is very different, won’t benefit from previous work and won’t be easy”.
All of this is necessary to gain a shared understanding, and the end result may still be that the work must be done.
Unfortunately, the part after the "can't you just" is rarely helpful.
If the “why can’t we do X” is “because we’re a team of 4 and already overburdened” there may not be any practical solutions.
Most of the time the issue is the "9 women can't have a baby in a month" problem where adding more resources is not going to make things happen faster - in fact it may slow things down.
But the business doesn't understand that software is not like construction where adding more people really will get that ditch dug faster.
At its core software is not making a thing - it's inventing a machine that makes the thing.
If you're Little Debbie and you have a machine that makes cupcakes it's hard when the business comes in and says, "now it needs to make fudge rounds." And no amount of extra people working on the problem will get that machine retooled any faster.
People tend to understand that there's an upper limit to the amount of people that can be deployed to dig a ditch any faster too
They just refuse to believe that with software, sometimes that number is smaller than they think it should be simply because people aren't physically getting in each others way
But it's super common for a team to be asked to do something fundamentally outside their scope, and the asking client/boss not realizing that they are doin g it.
"Figure it out" as a useful strategy sort of assumes that both sides of the equation are clued in about this and roughly on the same page.
"I underwrote a great team, you should be able to figure this out (stop whining about it being hard)" is a fundamentally different statement than "Why can't you just do X, how hard can it be?"
It's also worth remembering that the plural of "resource" is not "team".
> "figure it out"
that's the kind of motivation that leads to someone making Armchair Toilet
https://i.pinimg.com/originals/ce/11/34/ce1134ca396e43f55c84...
Sure, sometimes you can re-define the goal. ("We need to know more about what's in the Alpha Centauri system.") But sometimes domain expertise means telling non-experts about reality. (Pi is not 3, no matter how much someone thinks it should be.)
See: the full rewrite/retool
This resonates with me. I get a lot of "But Microsoft Office 365 Cloud does…"
I tell them that Microsoft spent a quarter of a billion dollars on that item last year, and I'm perfectly happy to re-create it if they give me a quarter of a billion dollars, too.
> Steve Jobs would just say "figure it out".
The difficulty in what my clients are asking for is often not the feasibility but rather the time, monetary, and opportunity cost for ignoring other features.
Apple didn't pay well for most of its life, until well after iPhone/iPad success
It was not like "FAANG salaries", which started around 2011, by my reckoning because Facebook didn't agree to the "Steve Jobs collusion" with Google/Intel/Pixar (ironically!)
This is very well documented
https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...
Steve Jobs e-mails Eric Schmidt (2007) -- https://news.ycombinator.com/item?id=28699873
Steve mostly realized the good engineers want to work with other good engineers, no matter what they are paid
He didn't actually give people "budget"
As a side note, engineers tend to be bad at couching conversations in these terms - nothing is impossible*, things just cost more or less, and that’s a decision for the money people to make.
* yes yes caveat caveat
Where I work, it doesn’t really matter what I’m paid, we don’t have a budget to go get what we need. If I think a certain tool would help get a job done faster, too bad, I have to cobble together what I can with the tools I’ve been provided. If I think something will take a year, too bad, it’s due in a month and we’ll have an hour long meeting every day to ask why it isn’t done yet.
In this context, employee pay is largely irrelevant.
He got the money together to buy one. Or thought he did. Forgot about insurance, being a young male and little driving history. Doubled his payments.
Also he was a terrible driver. The expensive thing about BMWs is not the car it’s repairs. Which is also why the insurance is so high. He wrecked that thing three times. He probably could have gotten his family into a better house for the amount of money he burned on that car. So foolish.
Customers live out this sort of drama all the time. They need a good used car but they want something that’ll bankrupt them because they have an image in their head. Either a dream or a status symbol.
Gordon Ramsay did a whole TV series about restaurateurs with the same mental block. They want to be successful but they don’t want to pay their dues and pretend instead.
I had a boss who I’m now sure had aphantasia and his two best UI people would tell him something wouldn’t fit in backlog grooming, then we’d get to spend a week showing him all the different ways it doesn’t fit before he would shut the fuck up about it.
We already “figured it out” in our heads because we are equipped to do the job. We were already doing some pretty sophisticated typesetting to make more things fit on the screen than really did.
He was sure he knew where the problems were in the org and that they weren’t him.
See also why estimates aren’t actual costs. Every plumber runs into this constantly. Sorry you have a crushed sewer line add a zero to the cost.
I'll be keeping this on file, thank you.
I think I'll use your analogy next time, "and then robotics is like gardening. You don't know when the weather will change what wild life will come and try to ruin it, or the soil composition. At least in the house, you can be certain everything is made by human for humans, and things make sense. Outside, not so much. "
Client: My family is having a house built right now.
Programmer: Oh, I'm so sorry for you.
The GIS took decades to be developped and become functionnal.
The tools allowing to do a GIS lookup also took decades to be developped.
The tools allowing to encode the geodata within a picture file idem.
Not mentioning the development of the necessary hardware to take the pictures.
At the time the comics was being written, GIS lookup had been made easily available since what? 10 years?
And 10 years later, another layer is now easily available - also after decades of collective research and development.
It's not about "the difference between easy and hard challenges in software".
It's about the maturity of a software and its ecosystem, and understanding how even some small incremental changes can have an important impact.