for a model to be useful, it doesnt need to capture the behavior of a system. It only needs to capture signals which can be useful. For example, for a biased coin toss, a model is already useful if it can predict a little better than random.
1,311 karma · joined December 10, 2024
for a model to be useful, it doesnt need to capture the behavior of a system. It only needs to capture signals which can be useful. For example, for a biased coin toss, a model is already useful if it can predict a little better than random.
Having worked extensively with vibe-coded software, the main problem for me is that I have tuned-off from the ai-code, and I dont see any skin-in-the-game for me. This is dangerous because it becomes increasingly harder to root-cause and debug problems because that muscle is atrophying. use-it or lose-it applies to cognitive skills (coding/debugging). Now, I lean negatively to ai-code because, while it seduces us with fast progress in the first 80%, the end outcome is questionable in terms of quality. Finally, ai-coding encourages a prompt-and-test or trial-and-error approach to software engineering which is frustrating and those with experience would prefer to get it right by design.
You read the words like they were generated by an AI - ie, empty words which did not make sense to you. ie, you did not derive meaning from it.
But for me, I derived meaning from this essay. Regardless of how the text was generated, I was able to relate it to some of my own insights.
For instance..
>Pascal used to distinguish between geometric and subtle minds (sensibilities). >The geometric mind works through axioms and deductions, step by step, like a >compiler. The subtle mind grasps a situation holistically, reading context, >feeling the weight of unspoken constraints, sensing what a room will and won’t >tolerate.
I read this as saying that as humans we have a "subtle" sense that can perceive the chaos out there and then derive patterns from that chaos and be able to symbolize them to derive theories of that chaos via the geometric mind. For example, the sense or feeling of space is symbolized into euclidean geometry. To me, this is a deep insight, and I did not know Pascal called it out. So, I learned something from this essay.
Among people there are 2 types:
1) Those that gain self-esteem by associating themselves with leader-like figures who project visions of the future. This explains cults, religious and political groups and the like. This comes from our ape-dna where the alphas are blindly worshipped.
2) Those with enough cognitive development and independent thinking who can avoid their ape-brain getting short-circuited into worshipping the alphas.
Majority of people are in group 1) and my hypothesis is that they form a vast-majority of the customer and employee base of musk cos.
> The fact that Euclid existed doesn't tell us much about what the other billions of humans are doing cognitively on a Tuesday afternoon.
Birds can fly - so, there is some flying intelligence built into their dna. But, are they aware of their skill to be able to create a theory of flight, and then use that to build a plane ? I am just pointing out that intuitions are not enough - the awareness of the intuitions in a manner that can symbolize and operationalize it is important.
> The whole point, drawing on Kahneman, is that most of what we call reasoning is fast associative pattern-matching, and that the slow deliberate stuff is rarer and more error-prone than people assume
David Bessis, in his wonderful book [1] argues that the cognitive actions done by you and I on a tuesday afternoon is the same that mathematicians do - just that we are unaware of it. Also, since you brought up Kahneman, Bessis proposes a System 3 wherein inaccurate intuitions is corrected by precise communication.
[1] Mathematica: A Secret World of Intuition and Curiosity
Humans can produce new concepts and then symbolize them for communication purposes. The meaning of concepts is grounded in operational definitions - in a manner that anyone can understand because they are operational, and can be reproduced in theory by anyone.
For example, euclid invented the concepts of a point, angle and line to operationally represent geometry in the real world. These concepts were never "there" to begin with. They were created from scratch to "build" a world-model that helps humans navigate the real world.
Euclid went outside his "training distribution" to invent point, angle, and line. Humans have this ability to construct new concepts by interaction with the real world - bringing the "unknown" into the "known" so-to-speak. Animals have this too via evolution, but it is unclear if animals can symbolize their concepts and skills to the extent that humans can.
Also, Brandolini's law. And Adam Smith's law of supply and demand. When the ability to produce overwhelms the ability to review or refute, it cheapens the product.
Incredible, how an entire religion has sprung up around AGI.
Seeing this phenomenon, a silicon valley entrepreneur get an idea with the following sales pitch:
"Turd-bars that will make you the fittest version of yourself , answer all your deepest questions, and take you to the promised land (mars)."
Surprisingly, the turd-bars sell well, and GDP rockets up. Meanwhile VCs with fomo are funding its competitor: the shit-sandwich.
Great point. AI remixes and rips-off existing code-bases in a manner that is impossible to attribute copyright violation making it legal. ie, Perfect cloning. In a world where cloning is legal, the engineering cost of product drops to zero. That is where software production could be headed. What remains is marketing/distribution/sales.
There will remain niches solving "hard problems" which cant be cloned, but those will be rare. Hard problems are where a lot of engineering complexity resides, involving interacting components for which there are no examples in training datasets to copy from. For example, a complex distributed system or hardware with multiple nuanced tradeoffs.
[1] https://www.scientificamerican.com/article/first-proof-is-ai...
Take a look at the history of the power loom which automated weaving in the 19th century. The number of handloom weavers dropped two orders of magnitude after the power loom.
IBM still sells mainframes but is no longer a growth darling.
> Markets are right to reassess multiples. But reassessing multiples is very different from pricing in extinction
What you are missing is that the SaaS companies were extremely overpriced. For instance, crm after all the carnage is still priced at 25 times earnings which is historically high for anything that is not a growth company. The perception was that these companies would print money year after year selling software trinkets on their platforms and as such were placed in the growth category. Now, it is plainly obvious that these software trinkets can be produced easily by anyone using AI. Their pricing-power has dramatically declined. Hence the re-rating. None of this contradicts the thesis in your ai-assisted article that these businesses have moats just like IBM and its mainframes. These businesses are now in a vicious reflexive narrative loop where the narrative will impact the real-world which will further fuel the narrative.
Automation of easy templated tasks will cause a huge disruption. Production of software used to be a skilled job, but now is automated to a large degree. This has huge impacts to the profession as a whole. Already, enrollment to the UC CS program is declining.
Are they rote-students imitating or copying memes and as such are driven by inadequate-ideas or are they students who understand the subject from its first assumptions and as such are driven by adequate-ideas. In the quote above, the suggestion is that majority are rote-students.
Majority of people are sleep-walking as machines driven by imitation, habit and external forces. We live in a dreamlike, mechanical state lacking the awareness of this itself. apropos: Gurdjieff
We are also limited by the linguistic structures we inhabit. And many languages have multiple variants. There is the respectful, obedient "formal" variant used at the workplace and the informal "colloquial" used in other places.
I would tend to disagree. The tech types have a strong intellectual center, but weaker emotional and movement centers. I think a realignment is possible with practice. It takes time, and as one grows older, the centers begin to integrate better.
I would disagree. Formalism and precision have a critical role to play which is often underestimated. More so with the advent of llms. Fuzziness of natural languages is both a strength and weakness. We have adopted precise but unnatural languages (math/C/C++) for describing machine models of the physical world or of the computing world. Such precision was a real human breakthrough which is often overlooked in these debates.
I learnt something from this thread.
That, respectfulness and politeness are more from intentions/actions than from speech alone. Politeness of language without any respect for the actual function of that speech is pointless. Indeed, that this what the LLMs are trained for. Form over function. And many humans get fooled by it and are also clueless like the person dropping the steaming turd of a PR.
I can see your frustration. You are looking for reproducible "benchmarks". But you have to realize several things.
1) research level problems are those that bring the "unknown" into the "known" and as such are not reproducible. That is why "creativity" has no formula. There are no prescribed processes or rules for "reproducing" creative work. If there were, then they would not be considered "research".
2) things learnt and trained are already in the realm of the "known", ie, boiler-plate, templated and reproducible.
The problems in 2) above are where LLMs excel, but they have been hyped into excelling at 1) as well. And this experiment is trying to test that hypothesis.
We will learn if the magical capabilities attributed to these tools are really true or not. Capabilities like they can magically solve any math problem out there. This is important because AI hype is creating the narrative that these tools can solve PhD level problems and this will dis-infect that narrative. In my book, any tests that refute and dispel false narratives make a huge contribution.
What do you mean ? These are top-notch mathematicians who are genuinely trying to see how these tools can help solve cutting edge research problems. Not toy problems like those in AIME/AMC/IMO etc. or other similar benchmarks which are gamed easily.
> that others (e.g. FrontierMath) already did everything they claim to be doing
You are kidding right ? FrontierMath benchmark [1] is produced by a startup whose incentives are dubious to say the least.
[1] https://siliconreckoner.substack.com/p/the-frontier-math-sca...
Unlike the AI hypesters, these are real mathematicians trying to inject some realism and really test the boundaries of these tools. I see this as a welcome and positive development which is a win-win for the ecosystem.
Agree. The article could have been summarized into a few paragraphs. Instead, we get unnecessary verbiage that goes on and on in an AI generated frenzy. Like the "organic" label on food items, I can foresee labels on content denoting the kind of human generating the content: "suburbs-raised" "free-lancer" etc.
Why the angst ? If the ai can autonomously solve these problems, isnt that a huge step forward for the field.
Great insider feedback. What is the switching cost for a SoR ? Is it embedded so deeply that is impossible to switch (like IBM mainframes). Wont the incumbents make a good living increasing per-seat prices each year to their required growth rates, and their customers have no option but to pay-up.