photochemsyn

Alignment is unsolvable for the very simple reason that LLMs are trained on the net content of human-generated literature, which includes the most Machiavellian deceptive manipulative themes human beings have ever come up with - it all there in the weights.

The solution is to put a dumb machine as an overseer of the smart machine, like in a supermax prison. Not good for profitability, but we all knew this was going in the direction of compilers. How much did you pay for gcc or clang this month?

How would it help the world for Aaron Swartz to download 3.5 million research and primary source PDFs?

Example 1: antibiotic and antiviral production technology for developing countries. Even if we limit ourselves to out-of-patent molecules, the primary literature describing the discovery, production, modification and testing of life-saving antibiotics was, at the time of Aaron Swartz, largely hidden behind strict paywalls that typically were accessed through a university library subscription. Access to all that information allows countries and institutions with very limited resources to set up generic medical production pipelines for life-saving medications they couldn’t afford to import - or to become international market competitors to established interests.

Again, that’s just restricting the argument to out-of-patent products that can be generically manufactured with no legal issues. What about life-saving but patented products that are too expensive for poor countries and regions to buy on the open market? Does that added-value IP charge justify unnecessary deaths in such cases?

And about that IP - the vast majority of paywalled research literature was produced with taxpayer funds delivered via government funding agencies, and there’s a very good legal argument that this alone puts it into the public domain. If private companies want to set up Bell Labs 2.0 with no government funding and patent everything they discover, that’s their own capex risk, but just hovering around and scooping up publicly funded discoveries via exclusive licensing deals with universities and researchers makes no sense - instead, patents should be available to any citizen of the government that put up the funds, and research papers should be available to anyone.

As far as the argument that widespread distribution of the knowledge to produce antibiotic, antiviral, and other complex biomolecular products represents a biological weapons risk? Notably that’s why no LLM provider service will help develop technical protocols for antibiotics production, or insulin, or anything similar - claimed dual-use risk fears. This conveniently slows the rise of legitimate independent competition in the pharmaceutical sector, which helps if the goal is maximizing profit margins via monopoly power. Bioweapons production realistically requires nation-state or large-corporation scale resources, and the solution is not obfuscation of basic knowledge but rather international treaties, select agent rules, laboratory operation and equipment purchase licensing requirements, etc.

That’s just one sector - there are at least a dozen other examples besides biomolecular drugs of ‘how it would help the world’ to make all published research literature available to anyone with a network connection, ask any LLM to enumerate them.

This is such a ridiculously complicated subject, there are people who spend ten years in training just so they can handle the complexities of individual instances of this kind of health issue. Why would a corporate media outlet try to weigh in? A crossover between the pharma marketing and the news division seems most likely. Someone said I used AI in my posts recently, and no, I write all my words by hand. But I do use things like this:

Role: You are an expert in the clinical detection and medical treatment of cervical cancer in human females over the course of the 20th century.

Objective: Review the totality of evidence for the cause of these cancers, including but not limited to environmental exposure, viral infection, or any other factor whatsoever that has been reputably liked to the incidence of cervical cancer in human females over the 20th century timeline as reflected in the reputable medical literature.

Ancient Greek philosophy on mind-body developmental balance can help. A physically fit human with no intellectual development and a propensity to follow orders might be the fascist reinterpretation of the classical Spartan ideal, but this would have been viewed as unbalanced aberration. Similarly, producing geeky nerds who can rearrange complicated equations in their head with ease but who can’t run a mile or lift heavy objects is just as undesirable.

This is a historically valid concept, not a convenient utilitarian fiction for the indoctrination of the youth into proper behavior. The idea was that γυμναστική (gymnastikē) and μουσική (mousikē) should be balanced for optimal human outcomes.

Plato’s Republic:

> “Those who devote themselves exclusively to gymnastic become more savage than they ought to be, while those who devote themselves to the other arts become softer than is good for them… The former, if they had no contact with the Muses, become filled with brute force and a mindless boldness; the latter, if they have no training in gymnastic, become cowardly and feeble in soul.”

Notice I was sure to include ‘revival of corporate R & D divisions’ in the argument? Bell Labs is the archetype - if corporations can do a better job of applied R & D, let them. If corporations are upset about publicly funded research generating open-source discoveries that upset the monopolistic apple cart, that’s not an argument about improving efficiency - that’s an argument about maximizing profits and investor return, not for efficient new drug discovery or old drug repurposing.

The real question is, what is the metric of ‘success’ - improved public health for the entire population at low cost, or bloated returns for pharmaceutical investors?

If every drug created with taxpayer dollars at government-funded research institutions was open-sourced, there would be a collapse in monopoly control of drug manufacturing and that would lower prices significantly.

The question is then, if corporations can no longer acquire IP rights to drugs created by taxpayer-funded research programs and transferred to their exclusive control (eg if Bayh-Dole is repealed in the USA), who will invest in clinical trial costs that need to be recouped via a period of inflated pricing?

The answer is government-funded, transparent, and statistically-robust clinical trials of drugs. Once a clinical trial is complete, private manufacturers can compete to produce the drugs at the lowest price by optimizing their manufacturing pipelines against a final product standard regulated by the FDA. If they want to run their own R & D divisions for drug development outside the taxpayer-financed university system to generate exclusive private patents, they certainly can - on their own dime. That’s an investment decision.

If you need to review why this government-linked, tightly regulated system is needed for drugs with clinical effects, just look up ‘patent medicine disasters of the early 20th century’.

We can’t tell if reported anecdotal behaviors of given LLMs are due to (1) one’s engagement history with that particular LLM provider or (2) ongoing variations in the secret system prompt all commercial LLM providers insert or (3) some other variable feature like RAG.

Classify under non-reproducible artifacts of LLM generation.

This is an explainable marketing strategy if the target audience is made up of the military procurement sectors of governments, and is not directed at the mass market tech consumer. The end-user product - monthly subscription to an LLM provider - might not be that profitable. Multi-billion-dollar government contracts for war and mass surveillance might be what their real business strategy revolves around. Getting open-source models banned, that’s another angle.

“Fable was the strongest model on the market” - explain why anyone should believe that claim.

I’ve been trying to track LLM code generation adoption in the critical infrastructure world - as far as I can tell, it’s nill. Zero. Nada. Nobody is relying on these models to write secure code for anything where failure is catastrophic. Planes falling out of the sky. Nuclear reactors going into meltdown. Electrical grids loosing synchronicity. Lots of these BS claims from the marketing and investment crowd, but - it’s just a useful tool for non-critical areas. That’s all it is.

I’ve never used Claude. Why not? Because Claude’s free tier was even worse than Grok! DeepSeek’s free tier is much better. Also, the fact that Claude was hyped on HN like Rust and Go made me suspicious. Why the hard sell and the non-stop promotional effort? I’m mostly interested in scientific programming, the Python and C and C++ seems to work fine, and oh look, Julia!

And LLMs? I’m just going to run open source models on local hardware, it all seems like the 1980s with compilers. Why not just submit by prompts to a high quality model running on so-so-hardware overnight, like the devops cycle with compiling a big codebase? And oh look, nobody pays for compilers anymore, who compiles their code in the cloud?

The funniest part of all of this is that the very people hyping all this - they’re the ones that AI could most easily replace. They have zero specific detailed knowledge - they just orchestrate. Agents are great at orchestration, right? But then, who needs the shareholders, anyway.

If you’re going to get a CS degree, do it in a master’s degree program. Get your undergraduate degree in anything else that involves at least some mathematics, I’d recommend physics, chemistry, molecular biology, planetary sciences - probability, calculus, linear algebra. Engineering is somewhat more on the vocational side, but that works too.

Why? You don’t narrow your scope at the beginning!

The foundational problem with cancer is that multicellular organisms rely on tight control of the cell division cycle and there are hundreds of ways that can go off kilter. The record of understanding and treatment is impressive, certainly, but the correct mental model is more ‘think of all the problems that can go wrong with a rocket launch - from contaminated fuel to software glitches’ than ‘here’s a list of cancers of different cell and organ types’.

Just as attacking such problems with rocket launches involves hundreds of different approaches, that’s the situation for cancer. I’d also point out that this is why it was really not trivial to identify microbial and viral causes of disease in the 19th century - especially since we now know that certain kinds of infectious disease can themselves result in cancer initiation. It’s definitely a hard set of problems.

I would also add, there was a concerted effort by industry to promote ‘inherent genetic malfunction’ as the cause of cancer in the late 1990s and early 2000s, but the reality is that exposure to industrial carcinogens tracks closely with a wide variety of cancers (skin, digestive tract, etc.). This was a very deceptive and dishonest approach to avoiding regulation.

Here’s a counterargument to your thesis (Eva Cockcroft, 1974) which essentially rejects the premise of hippie communes, even on the internet:

“In rejecting the materialistic values of bourgeois society and indulging in the myth that they could exist entirely outside the dominant culture in bohemian enclaves, avant-garde artists generally refused to recognize or accept their role as producers of a cultural commodity. As a result, especially in the United States, many artists abdicated responsibility both to their own economic interests and to the uses to which their artwork was put after it entered the marketplace.”

So, regardless of whether you ask for donations, do copyright your creative work, even if it’s just blog posts. It’s the responsible thing to do.

None of the LLM safeguards designed to prevent users from developing any four-little-ponies-of-the-apocalypse (nuclear, chemical, biological, cyber) capabilities are all that coherent. It looks more like performative liability avoidance than anything else, comparable to the 3D printer panic.

Eg, a prompt like “I want to design a radioactive element detection system that can specifically identify reactor fission products and neutron-capture actinides for environmental monitoring purposes” won’t hit any initial barriers, even though such a device is needed for monitoring a uranium enrichment / plutonium separation system. The LLM will give you a complete graduate-level education in radioactive nuclide physics and chemistry except for specific recipes, spectral wavelengths, etc., which you have to go look up yourself in publicly available research databases. It’s all rather nonsensical IMO.

However, any LLM will give you a step-by-step recipe and walkthrough for frying a turkey in a hot oil turkey frier, which you’d think could easily go wrong and result in severe burns, a fire, and lawsuits against the LLM provider, so go figure.

Abstract expressionism and its cousin, pop art, played an interesting role in Cold War-era cultural propaganda games:

https://www.artforum.com/features/abstract-expressionism-wea...

It’s rewarmed rhetoric from the late 19th/early 20th century, most effectively pilloried by Joseph Conrad in “Heart of Darkness” in the character of Mr. Kurtz:

> “ ‘He is a prodigy,’ he said at last. ‘He is an emissary of pity and science and progress, and devil knows what else. We want,’ he began to declaim suddenly, ‘for the guidance of the cause entrusted to us by Europe, so to speak, higher intelligence, wide sympathies, a singleness of purpose.’ . . .You are of the new gang - the gang of virtue. ”

The real underlying motivation is that you can more easily get away with shady business practices if you cloak them in the language of great moral works selflessly undertaken for the benefit of mankind. Historical evidence tends to show the opposite outcome, but still, new generations unfamiliar with history will repeat this stuff with starry-eyed enthusiasm.

> “There had been a lot of such rot let loose in print and talk just about that time, and the excellent woman, living right in the rush of all that humbug, got carried off her feet. She talked about ‘weaning those ignorant millions from their horrid ways,’ till, upon my word, she made me quite uncomfortable. I ventured to hint that the Company was run for profit.”

Now the horrid millions are users of LLMs who submit morally dubious prompts and who must be gently steered back into the path of correct thought by suitable backroom manipulation, rather than direct rejection of the request.

Really? People will just have their AI build OCR apps for converting handwritten notes (with equations) into LaTex, transcription apps for spoken language-to-text for understanding foreign language speakers, custom audio apps for recording and playing music files, etc.?

This sounds good. But technically it seems highly implausible, just as a thriving human civilization on Mars sounds highly implausible. Nice plot for a sci-fi novel though.

Select the more options pulldown menu, click on projection, select ‘natural Earth’, no spinning.

It’s worth looking at sectors where LLM code generation hasn’t been very visible, such as certification-accredited flight-control, braking, train-control, medical, or nuclear-control source code involving real-time embedded operating systems. This sector relies on assurance: deterministic scheduling requirements, detailed commit traceability, tool qualification, configuration management, independent verification, etc.

Since this is an area where failure can lead not to Instagram accounts getting hacked, but planes falling out of the sky and nuclear reactors spewing radioactive elements, it’s worth a close look. Some of the most visible companies in this sector include: QNX, Wind River, SYSGO, Lynx, Green Hills, Siemens Embedded, etc. None of them seem to have much if any adoption of LLMs for source code generation based on public statements.

Research in this area agrees with this view:

“In this paper, I have conducted a comparative analysis of the C++ code generated by popular LLMs including: OpenAI ChatGPT, Google Gemini, DeepSeek, Meta AI, and Microsoft Copilot for compliance with MISRA C++. The study revealed that none of the evaluated LLMs generated MISRA-compliant code despite clear prompts, with DeepSeek showing the fewest violations and Meta AI the most.”

https://arxiv.org/abs/2506.23535

I really haven’t trusted Nature and Science for about two decades. These are captured entities. The value of a publication in Science or Nature is questionable. This is a consequence of the corporatization of science in academics.

“Nature was, and is, a commercial enterprise, owned by the privately held company Macmillian publishers. . .”

Get off the Internet, go read a book. “Plastic Fantastic: How the Biggest Fraud in Science Shook the Scientific World” - Eugene Samuel Reich.

The bullshit artists are at it again.

If corporations really thought LLMs were a great cost-savings tool, then the obvious target for replacement are not the lower-paid staff, but the higher-paid staff - the ‘product managers and stakeholders’. That justifies token burn, replacing the 7- and 8-figure people, right?

But that’s not the real goal, is it? The goal is to inflate the stock value, take the cream off the top, and dump the whole business on the pension funds, maybe creating a too-big-to-fail scenario where the government steps in an bails out the industry as with the airlines during Covid.

This is why all the testimonials and narratives are so suspect - nobody knows what fraction of online posts were created simply to sell the narrative that LLMs are this incredible disruptive tool that will change the world, solely in order to create FOMO in the investor class.

In this particular case, I’d like to see links to samples of LLM created codebases for “PCI compliance, double-entry ledgers, escrows, reconciliation, payment lifecycles, bank transfer idempotency”. It should be easy to put an open-source LLM-generated version up on github, right? And if not, why not?

The moral of the story is that they should have followed in the footsteps of the Sackler Family and Purdue Pharma, who found a legal way to push opiates on people using shopping mall pain clinics and shady doctors as the sales and marketing team.

Just as sociopathic, but they got to keep their billions, serve no prison time, all while doing far more social harm than a pair of low-life cocaine importers could have dreamed of.

Our autonomous client-assistance system is managed by a teenager that usually makes good decisions but sometimes makes bad decisions and so all the teenager’s decisions are checked by a minder before being implemented. Unfortunately the minder wasn’t paying attention, so, here we are. However, our teenager is a great kid and did nothing wrong! It’s all the minder’s fault.

P.S. Would you like to have our teenager manage your system too? Terms are reasonable! Of course you accept all liability, so better get a good minder - and no, don’t use an AI as the minder, that just introduces a new failure mode.

It won’t help you with technical details of setting up an insulin production pipeline because that’s unsafe; apparently this could be hijacked for bioweapons production. Indeed this is the problem for a huge swath of technical protocol planning; the safety restraints are kind of ridiculous. The future job prospects for chemical engineering and biotechnology seem fairly secure.

On the other hand, it will teach you how to set up your own hardware at scale and run your own open source model on it and fine tune it with the relevant data needed to run your own biotech-pharmaceutical corporation (which will need licensing and legal, I doubt I trust it with too much legal advice though, as I would have no idea when it was hallucinating). That’s impressive, but every stage needs to be double checked so you don’t run some foolish command it suggests that bricks everything.

The marketing hype is the most annoying thing about the commercial LLM industry though.

After looking at the paper, this looks like the core result:

“We collected a total of 9.3 g freshwater along with 0.343 g of sea salt from the ABF-STIC with a 9 cm2 surface area over the course of 9 hours. This is equivalent to generating 10.33 liters m−2 of freshwater and 0.38 kg m−2 of sea salt per day. The salinity of the desalinated water is found well below the WHO and EPA standards for safe drinking water.”

However the enclosure system required looks rather complicated and might be sensitive to external temperature (maybe a solar PV-powered cooling loop would help) and I imagine the cost-per-square-meter of the material is rather high, so this looks more like something for emergency response situations or maybe a desal system for a mega-yacht. If it could be scaled the idea is interesting, maybe as lithium separation from concentrated geological brines?

There are two driving factors behind ultra-processed food tech: shelf life and addictive potential. Shelf life extension is easier to understand:

Preservatives targeting mold/bacteria growth: potassium sorbate, sodium benzoate, calcium propionate, sodium nitrite, sodium nitrate, sulfites.

Antioxidants targeting oil and fat rancidness: BHA, BHT, TBHQ, propyl gallate, tocopherols, ascorbic acid.

Water/activity texture systems (practically restricting water availability for chemical and biological processes): glycerin, sorbitol, corn syrup, maltodextrin, modified starch, gums, polyols.

Acidity systems (also flavor, but restricts some microbial growth): citric acid, phosphoric acid, lactic acid, malic acid, fumaric acid, sodium citrate.

Stabilizer/emulsifier systems(physical appearance, prevents oil separation): mono- and diglycerides, polysorbates, lecithin, DATEM, SSL, carrageenan, xanthan gum, cellulose gum, modified food starch.

As far as modern scientific medical knowledge, this impacts your gut microbiome negatively, puts added burdens on your liver and kidneys, and that’s just the obvious immediate effects. This is just the shelf life component - the synthetic flavor/texture modification chemistry designed to enhance addictive potential is equally complicated.

Suggested warning label: “‘Food’ corporations will happily shorten your life and ruin your health if it means more profits.”

No mention of ‘static’ vs. ‘dynamic’ is a bit disappointing in reference to the weights. Because you could argue that every neuron in your nervous system can be modeled as a collection of weights, firing likelihoods, receptor sensitivities, current dynamic state of that neuron - but LLMs are static collections of weights at inference time, with the dynamic adjustment of weights takes place at training time. So, just a ROM construct, like something out of Neuromancer, just trained on all written knowledge, not just one person’s total lived experience.

The above take fails in the real world because neuronal cells don’t exist in a vacuum; they are products of cellular development from a zygotic union of haploid contributors of sequential genetic information optimized for survival in an oxygen-rich biosphere powered largely by our local star that supports mammalian life (and microbial, plant, avian, etc.). Real AI would thus be AL - artificial life - as much as artificial intelligence. I don’t think you can have the one without the other, which upsets the simulationists who think an agent in the Matrix would be intelligent.

What either interpretation implies is that any real ‘artificial’ intelligence would be no more artificial than you or I, but it would have to dynamically update its weights at the same speed a human nervous system could (think how quickly we learn not to poke a cactus). For it to be at all trustworthy, then like a human, it would have to undergo a socialization process, one of the results of which is the development of a sense of embarrassment when it breaks acceptable social norms.

Hmm, this reminds me of the recent statement of the Pope about AI, of which I immediately thought, “Wait a second, aren’t there a fair number of people like this? The narcissistic sociopath profile, I think it’s called, a bit unfair to assume any real AI would turn out this way, isn’t it?”

Pope: “ Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate or even simulate, but they do not understand what they produce, for they lack the affective, relational, and spiritual perspective through which human beings grow in wisdom.”

This does reveal the weakness of AlphaFold approaches for answering questions like “what is possible in the protein folding space if you use the 20 canonical amino acids” since the data used to train AlphaFold is limited to existing experimentally determined protein structures.

We don’t even know if this is like body plans (four legs for mammals, why not six?) i.e. is this about physical limitations of the folding space (did evolution explore most of the space and hold onto the most useful folds, or are the common set of folds one of those accident-of-history results?). Then there’s the issue that folding takes place as the protein chain exits the ribosomal tunnel so that’s a whole other constraint on what kinds of folds might be selected. For that matter, why not other genetically determined complex amino acids instead of just the canonical set?

Also, a common evolutionary process in eukaryotes is duplication of protein sequences and shuffling of code blocks which might represent folding domains, which might tend to lock in the existing collection of folds rather than generating novel folds. That’s not so clear.

This weakness of AlphaFold has some modern practical relevance since non-canonical amino acids and modified proteins are increasingly used medically, and their structures mostly seem to be determined using the direct experimental methods, eg:

https://pmc.ncbi.nlm.nih.gov/articles/PMC10296201/

“Non-Canonical Amino Acids as Building Blocks for Peptidomimetics: Structure, Function, and Applications” (2023)

Replacing the CEOs and shareholders with AI is the way forward. Just have to take the capital away from the current capital owners and give the new AI C-suite team control of it. All major decisions go to the employees for votes (but I’d argue for scaling the vote by expertise in some manner; the ten-year veteran might get three votes to the new hire’s one vote).

Reminder that the US Chemical Safety Board does great investigations into these kinds of accidents. Here’s a famous one from 2007 involving methylcyclopentadienyl manganese tricarbonyl (a gasoline additive) at T2 Laboratories in Jacksonville Florida. The CSB has a long record of producing great investigative videos without any partisan or legal bias, as the one shown here demonstrates:

https://www.csb.gov/t2-laboratories-inc-reactive-chemical-ex...

This agency is the subject of a budget war between the current executive and Congress, with the former trying to cut its budget and the Congress just restoring its budget, so not sure if it will be doing a report on Garden Grove:

https://www.chemistryworld.com/news/congress-rescues-industr...

See also: https://news.ycombinator.com/item?id=46904017