dsign

Well, it’s at least you and me thinking that.

> What would such a world look like?

Major religions, minor religions, cults will have a field day. Or more like a field century or a millennium or twenty, who knows.

Up to a few years ago, I would see or read some science fiction that was medieval times with spaceships (ala Dune) and I would roll my eyes and think “ffs, can’t people move on and invent something more original?”. Now I seriously believe that’s one of the most benign futures waiting for our species[^1]. Not that there aren’t lots of lights in this tunnel, it’s just that its shadows are very black.

[^1]: https://w.ouzu.im/aweb/unhinged

Kind of shocking that a 1980 chip could do 80 bits floating points. It makes me a little mad at all of those microcontrollers from this year, 46 years later that ship without floating point support or, at best, 32 bits, but they do come with a "secure island" and a radio capable to ring ET.

> Also, you still have to convince the public.

In terms of working democracy and social governance, we can kill a big flock of birds with a single shot if the public can understand cryptography. Think about it: we live in an age where everything is digital and has a microprocessor inside. Even money is digital. But the "public" can't parse simple logical statements, and they vote for the funniest clown in the race (for a definition of funny). The best this schism can possibly end is if we vote in a Luddite Empire that squashes technology. All other scenarios are far scarier.

Microsoft should stop tormenting developers and fix text input in MS Teams, then go and atone for its many, many sins, before committing new ones, like this “this not really an open weights model but we want to IE6-them with our bully tactics”.

There was a time, not too far in memory, when the value of being human wasn't about our intellectual prowess[^1]. We were creative with the human worth thing: being there in the time of need, faith, nobility or being a well-dreaded, literally-bloody viking full of beads in your beard.

Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.

[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.

[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged

Aye, will we stop hammering nails on God's coffin?

What's next? Maybe in a little bit we will discover that we H. Sapiens suffer from severe polysemanticity in our wetware because our individual neurons are messy and massive like an ENIAC while the corvids in the park have neat and efficient little brains that don't need unfolding ?... How's the pope (may God stop afflicting him with cancer or may Man make him live forever) going to explain that?

> By your logic there is no cancer problem till the heart disease problem is solved (which is well above cancer)

That's one of those things that causes me depression and anxiety: in many places cancer is actually above heart disease. From my vantage, that's a bad thing because people that die from sudden heart disease or from the sudden disintegration of their brains as the bullet from their gun travels through it get a better deal than those who die from cancer.

Suicide is sometimes a good solution to a problem (cancer) with no other solution. Make no mistake, I think that people should live forever even if we have to boil the oceans to get there; yes, I'm that kind of monster. But I'm not the kind that says that people should die in horrible agony just because of the stigma that doing otherwise imply.

Agree with everything else you said.

Things that work for depression: writing, composing music, exercise, playing video games. Things that don't help: alcohol nor drugs. Role of family and community: YMMV. I envy lots of folks with supportive and kind families, but I also witnessed much suffering and terrible loss caused by a loving family that doesn't quite get the loving part right.

With that out of the way, I'll say that for as long as suicide mortality is well below cancer mortality, society doesn't really have a suicide problem.

NPC is not rare even among not Asian-descent people, it’s just poorly documented; I lost my brother to it in July and we don’t have anybody coming from Asia our ancestry, that we know of. The doctor did said “there are only 20 cases of this in the world” after citing a paper where the sample size was 20. He did not order genetical tests nor tests for biomarkers for modern drugs. “Standard of care” my brother got in the US is from the 80s and it was unconnected to the molecular biology of his cancer (there has been a lot of relevant research in the last 20 years), but the doctors will subject you to it before even allowing you to access the better, most modern options. In my brother’s case, the standard of care treatment put fuel into his process instead of slowing it down, and by the time immunotherapies and other options were considered, it was already too late.

One thing I noticed is that the oncologists managed my brother far more than they actually worked on his case; oncologists are often overworked and, by necessity, desensitized. They are aware of new treatments because the pharma industry takes good care of that, but they are way less willing to read scientific literature relevant to a specific type of “rare cancer” present in a “single patient”, and they actually need the two things to be effective.

I feel for you and for your family; may the stars align for you and you get to be out of your crucible with the least amount of suffering.

I doubt that we will get there in my lifetime. Science is one thing, and maybe AI will super-charge some of it. But experiments will need to be made, and that needs brick and mortar labs, and labs need money, and at least right now, the absurd amounts of investments are not going in that direction. Oh, and did I mention regulators? They are very, very slow.

There could, however, be some promising signs down the road, like more capable in-silico methods, which will make it possible to bypass some/more of the brick and mortar labs. Another sign to watch for are orders of magnitude more companies and cartels of individuals doing advanced bioengineering outside of USA.

As our human labor loses economic value (and for realizing some of that shift the AI companies must brainwash people into believing that their AIs are human-like) people will reach for other sources of self-worth. Religious philosophy, and being made “in the image of God” (imago Dei) is a prominent one. What I find more than a little disquieting is that the AI companies seem to have grasped the relevance of this future playing field ahead of the rest of us. I hope these troublesome thoughts are a figment of some feverish dream of mine[^1] and nothing else.

[^1]: https://w.ouzu.im/aweb/unhinged

As others have said, no will needed for that. But (a simulation of) will is on my predicted timeline, since it is required for certain kinds of money-making endeavors.

> I think we all get that the % of jobs not doable by AI and robotics approaches 0 over time.

I also think that will be the case. However, not all the jobs are worth being automated. Say, if you are a modern-day horse-riding instructor, I very much doubt the robots will come knocking the door to take your job. Not because they can't, but because the headache is not economically worth it.

> However, it is now the case that AI can output mathematical arguments in situations without the human who prompted it being able to understand the arguments, verify them, or take responsibility for them.

I read this as "anybody can prompt, few can understand". And "we need more who can understand". If we had more mathematicians (than we have today) all of them piloting advanced models, the pie would grow. The problem is that AI capabilities drain (by disincentivizing) the education pipeline that would get us those mathematicians, and if recent rumbles about what AI is doing to education are to be believed, it does so many years before students even get to grad school.

IMHO, it is not that bad. Not having any human who understands linear algebra after the Butlerian Jihad is a win :-) .

> ... it doesn't feel like the progress is for me or the people I care about.

AI is doing very little to improve the human condition; all it seems to be good at is optimizing productivity and generating profits somewhere maybe. That is not an AI-generated problem; using profits as an all-things compass is a disease we have been cooking for ourselves for a long time. But I'm sort of hoping AI will make the problem so bad that we will be forced to fix it, or perish.

Plan mode is still very useful to me, I just need to read the plan. Even if the model is smarter than me, it’s still working on my project, and thus I need to keep a mental view of the system. My solution has been to ask the assistant for far less than it can do and leave for myself a bigger bunch of the work, so that I’m forced to keep myself up to date on the project. Yeah I’m leaving productivity on the table but… am I really?

To be fair, if VTE maintainer causes you to have a miserable day every day because your terminal manifests an annoying behavior, then you may feel justified on hurting VTE maintainer’s feelings. It’s like sitting in a torment chair with your hands and feet tied and trying to insult your tormenter into stopping the pain… okay, I can see how that may not work…

I’ve been developing a guitar pedal firmware in an ESP32-P4, and the flash image size is still under 1 MB and the PSRAM sits unused. It has not only a bunch of sound filters, control code for ADC, DAC a couple of USB devices, SDIO and LittleFS, but also a full-blown serial terminal TUI to control all sorts of details and to monitor performance. Said performance is good, but what blows me away is that there’s no jitter in the number that measures the number of cycles the processor executes while processing a sound frame, i.e. no preemption by the kernel to go and take care of its business.

There’s no way a multi-user OS such as Linux can be as efficient as a firmware blob that does one thing. Tell me if I’m wrong, but in my mental model, one can get about the same mileage with 16 MB of PSRAM in an MCU running FreeRTOS as one can get in a 512 MB SBC running Linux. For my use case, the winner combination would, of course, be a 512 MB SBC running FreeRTOS, but I’ve so far been unable to find such a wondrous thing.

> some lower form of intelligence

Anecdotally, but I have lived in different cultures with entirely different languages and/or dialects, and the thoughts and even entire categories of thoughts people from these cultures express, or can easily express, are very much shaped by their language. Relatedly, I've also often witnessed multilingual people switch out of their native language to a second one just to express a particular idea or nuance, because they can do it with two words in that other language but would need at least a couple of sentences to say the same thing in their native one.

We use formal language to express symbolic relationships, e.g. "A implies B". But even "A implies B" has multiple meanings: material conditional, strict implication, logical entailment, etc. So, symbolic systems are not "pure and hard", they are also contaminated and softened by the vagaries of language outside them, which is our primary access to those systems: "valid" natural language and its strings of words. A statistical system that can string words into valid(=allowed by the distribution) language asymptotically approaches reason. So, the mind is not in the words, but in the laws that permit many words to come together, i.e. the probability distribution.

Sadly, it happens here as well: https://cybernews.com/news/denmark-arrest-lars-andersen-doxx... ...

"Boring": that's a straw man, it was never in my comment. I did say that people should stop grumbling and use the tool for good things. I'm not quite sure what was your point, but it sounded like you disagreed (one doesn't deploy the straw-man weapon otherwise), and this "Without the human intellect to emulate, LLMs would be nothing" sounds like an expression of grief, which admittedly is a very reasonable feeling for a number of reasons. What confuses me is that creating an intellect at the human level or above is indeed a great human achievement, but it's not a good thing because...? Maybe an emulation of the thing is not as good as the thing, was that your meaning?

Here, let me add my own little straw man: "if God didn't make us, how exactly did humans get so smart? Is not from a) being created in His image and b) being given His love? Without God, we would be mud from the ground (and a rib!)." If you agree with this statement, then, understandably, everything we have done since the Enlightenment has been to secure our place in hell, and LLMs are a particularly impertinent such device, but compared to a true human mind like a figurine of clay to a true person.

But, if of all things the measure is Man, of the things that are, that they are, and of the things that are not, that they are not, and Man keeps measuring the thing and the thing indeed seems very smart, then Man should stop moping and deal with the consequences of their actions: either ban the thing or use it for their own benefit.

Upvoted. I think that said humans should get used to the idea that their intellect is special only insofar as it was first, and if they want to adopt or invent a value system that keeps them at the center, that's actually fine and good for their survival. But said humans would do well to stop laboriously and curmudgeonly moving the posts that mark the goals, and get busy trying to coax the AI to do useful things for them and for the species as a whole.

> To take Coxon's argument seriously, you have to believe frontier models are hypercapable, and not just hype objects.

It's harder and harder to make the point that frontier objects are just hype objects though. And, even if they stay where they are today, that's more than enough to make them incredibly...destabilizing, to say the least. It's not only it, but frontier models act like knowledge mixers and percolators. Some people (not necessarily everybody), are adept at the discipline of learning, building things and executing projects, and that's all you need today with LLMs filling many of the gaps such person can have. Now, most of those people are actually engineers. So I'm going to go with the article and say that we must stop selling our souls for money.

This is the tech at Cloudflare of course, but I swear that my first interpretation of the title was "We have replaced all our python coders with AI and they (the coders) are out and generally available" :-)

Your statement piked out my interest and I went searching for what Rubisco was. My prediction is that this particular one will be cracked out soon, and at around 2045, creating disgustingly efficient variants of RuBisCO along with other photosynthesis mechanisms will be AGIs’ hot hobby.

That there are no more efficient variations of it in nature just tells us that the local minimum is really deep and that natural evolution, as it is, can’t produce anything better, not even with a billion years and 10^30 organisms serving as a “brute force lab”. It’s also the kind of problem an AGI system would tackle for purely ideological reasons, i.e. to prove that it is superior to nature.

Roughly, there are two types of code in a user application: logic and media related. There are many exceptions, but logic code generally doesn’t need a lot of CPU. Media algorithms on the other hand are all CPU hogs, and SIMD helps there quite a bit.

Now, I also happen to think that modern desktops run a lot of background survtech…some from the operating system itself, but most from third-party software and web pages. So, in addition to more CPU performance, I would also like more system utilities that spy on the spies and automatically puts them in some public score of shame.

> In your first draft, most of your paragraphs are bad, your topic flow is incoherent, and you’ve got at least 750 words you don’t need.

I stopped reading at this point. All of the above may have been correct five years ago, when there was a "canon" of good writing, but now LLMs pack that canon as tight as teenagers in a boy band concert. So, go with a flow that reflects your state of mind--LLMs don't have that--and yield the words in fistfuls and with abandon. To further distance yourself from LLM output, your writing can be any or all of these things: sassy, prosaic, heathen.

Uses for LLMs: grammar checks, and to see if your idea comes through. However, depending on what you are writing, I recommend leaving in your writing a tiny doses of typos, grammar issues, and infuriating word order choices. Even some level of ambiguity is fine; your fellow humans and LLMs are not all of one mind, and if some of them are going to fight with knives over your exact meaning, well, it's them who are the extremists, not your problem.

Great article.

I often think of what we are buying ourselves with AI. Five years ago, when this wave started, my conclusion was that either humanity or humanism is scheduled for extinction. I love my species enough to prefer the lesser evil of the demise of humanism, if for that price we can save our species.

> The moment someone figures out a new modality for LLMs is when we’ll see the next hockey stick.

Honestly, that fills me with fear. LLMs exist to make money to their companies, and said companies are not gonna turn around and say, "you know what, go are going to make an android for each elderly person, that can not only help them with their medications, but that can actually make their medications, tailored to their biologies." Instead, they are going to go for the low-handing fruit of "you know Bob, the guy who makes jokes in meetings but who is grumpy about delivery timelines? Well, we are going to make an android to replace Bob. MetalBob will make even better jokes. The blue model will be able to explain in excruciating level of detail why timelines aren't reasonable. The red model will walk through the cubicles with a whip to ensure everybody keeps working all the time, and nobody goes to pee."

Though, on second thought, "government X could force Apple to disable feature for members of group Y" seems possible. I'm sure you can come with "Y" quite easily, heard anything about Ed Sheeran's tour?