595 karma · joined November 18, 2014
Find me at http://shonfeder.net/
Regarding LLMs specifically, my view is that the current state of LLM-based AI is leading us more towards '"write" our thoughts "in water"', because
* what we write to LLMs does not catalyze a transformation in understanding of them as systems, nor drive them to evolve in a sustainable way towards truth,
* and their outputs are the result of potent patterns in the turbulence of humanity's discourse, rather than an understanding or systematization that could "teach the truth adequately to others"
While I think their potency cannot be questioned, their sustainability and correctness certainly can be. IMO, AI in the vein of theorem provers and semantic webs are much more in line with a text that can actually speak back with integrity and genuine responsiveness, "teaching truth adequately". But I also guess
* the LLM profusion will likely catalyze some important developments and advances in those other approaches (and some use of LLMs will persist even after advances on a difference basis take over) and
* probabilistic methods are here to stay and will likely be more integrated into symbolic systems and
* we are at real risk of totally loosing the plot, or perhaps reenacting the plot of [Echo and Narcissus](https://en.wikipedia.org/wiki/Echo_and_Narcissus#Story), and seeing the warning of utter thoughtless wrought by irresponsible writing (as claimed in https://hedgehogreview.com/web-features/thr/posts/platos-war...)
There is another charge against writing in the Phaedrus, which is that texts cannot answer for themselves (contrary to spoken interlocutors) or adjust their communication to the needs and character of the reader:
> when they have been once written down they are tumbled about anywhere among those who may or may not understand them, and know not to whom they should reply, to whom not: and, if they are maltreated or abused, they have no parent to protect them; and they cannot protect or defend themselves
An interesting thing about computation is that it is a writing and re-writing that can indeed be responsive and, in a non-trivial sense, "answer for itself". LLMs do this in a form that is obviously very successful for engagement (and for hype) and for producing a lot of compelling output.
I think the Phaedrus also has something to say here:
> Soc. Then [they who who knows the just and good and honourable] will not seriously incline to "write" his thoughts "in water" with pen and ink, sowing words which can neither speak for themselves nor teach the truth adequately to others?
>
> Phaedr. No, that is not likely.
>
> Soc. No, that is not likely--in the garden of letters he will sow and plant, but only for the sake of recreation and amusement; he will write them down as memorials to be treasured against the forgetfulness of old age, by himself, or by any other old man who is treading the same path. He will rejoice in beholding their tender growth; and while others are refreshing their souls with banqueting and the like, this will be the pastime in which his days are spent.
> yet I have no doubts that this looping future is going to be our future despite the fact that I presently resent it
Why would anyone concluded this? LLMs are just one kind of application of MLs to software production. There is a vast solution space for automating parts of software production. The idea that slop loops are the inevitable future because they happen to be accelerating output at the moment just seems profoundly short-sighted and lacking in vision.
Brogue looks like this: https://syltefar.com/screenshot/?id=624
>
> It is with great joy that I have accepted the invitation to write you a personal letter. Even in your scientific writings your presence as a person is very strong, which is unusual for a mathematician. The traces of your personal life and your dramas extend beyond the limits of your own personal circumstances to concern us all ...
>
> ...
>
> Perhaps the greatest catastrophe of anti-scientific computationalism can be seen in the recent theory of "The End of Theories". In a series of widely quoted articles, informaticians or managers of very large databases explain that: "Correlation supersedes causation, and science can advance even without coherent models, unified theories". In short, networked computers, bringing to light very extended correlations in huge databases, make it possible to predict and act, without the need to "understand": scientific intelligibility is an uncertain luxury that is subjective and outdated, and theories are fallible proposals. Data, especially in large quantities – tera-terabytes, Big Data – is objective, is a new form of the absolute, is individually exact, expressed in digits. Thus, they argue, the larger that databases become, the more that statistical regularities, brought to light by computers, can govern us without the need to understand the meaning of the correlations, to interpret them, and without the need for theories about them, for interpretations.
>
> Fortunately, mathematics allows us to demonstrate the absurdity of these claims: Cristian Calude and I have written an article about this. Precisely the immensity of data involved has allowed us to apply the theorems of Ramsey and Van der Waerden. These make it possible to show that, given any "regularity", or any correlation between sets of numbers, you can find a number p large enough, such that every set with at least p elements contains a regularity (or a correlation between numbers) with the same structure. Now, since this applies to every sufficiently large set (with at least p elements), this also applies when it is generated … by a random process. Indeed, we observe, almost all sets of fairly large numbers are algorithmically random (one can give a mathematical definition of them, in terms of incompressibility), i.e., the percentage of non-random tends to 0 as p goes to infinity. So, if you observe regularities in increasingly large databases, it is increasingly likely that the inserted data are due to chance, in other words are perfectly without meaning and do not allow prediction nor action.
>
> ...
First we develop the machines, then we contort the entire social and psychic order to serve their rhythms and facilitate their operation.
(Caveat that I don't claim to be a λProlog or expert.)
All examples showcase the typing discipline that is novel relative to Prolog, and towards day 10, use of the lambda binders, hereditary harrop formulas, and higher order niceness shows up.
Fun times are coming.
When me move from just making use of something to using something to make with, that is when we should have a deeper understanding I think.
Does that sound right?
> the important distinction here is that I wouldn't call myself an electrician if my relationship to the subject matter doesn't extend beyond the desire to flip a switch.
Yeah, that seems right to me!
> I do want to make things, and many times I dont want to know something, but I want to use it
This confesses the desire to make, to use, and to make use of, without ANY substantive understanding.
Of course this seems attractive for some reasons, but it is a wrong, degenerative way to be in the world. Thinking and being belong together. Knowing and using are two dimensions of the same activity.
The way of these tools is a making without understanding, a using without learning, a way of being that is thoughtless.
There's nothing preventing us from thoughtful, rigorous, enriching use of generative ML, except that the systems we live and work in don't want us to be thoughtful and enriched and rigorous. They want us pliant and reactive and automated and sloppy.
We don't have to bend to their wants tho.
It's true this a matter of taste, but also worth noting that the OCaml compiler devs have made it very clear they are open to well-motivated extensions of the stdlib, and it has been growing at a decent clip in the last few years.
> Large Language Models represent a fundamentally degenerative technology because they systemically devalue the very processes that underpin human progress: original thought, rigorous inquiry, and shared trust. On an individual level, they encourage cognitive offloading, substituting the difficult work of critical thinking and creative synthesis with effortless, probabilistic text generation. This fosters an atrophy of intellectual skills, making society more dependent on automated systems and less capable of genuinely emancipated thought. This intellectual dependency, in turn, threatens long-term technological advancement by trapping us in a recursive loop of recycling and rephrasing existing knowledge, rather than fostering the groundbreaking, first-principles discoveries that drive true progress. Ultimately, this technology is dangerous for society because it erodes the foundation of a shared reality by enabling the mass production of sophisticated misinformation, corroding social trust, and concentrating immense power over information into the hands of a few unaccountable entities.