manofmanysmiles

I imagine some of the maintainers of distributions are uncomfortable with the amount of LLM-generated code.

I also imagine they will be left behind, for better or for worse as AI-authored software and hardware will satisfy people's needs faster than that developed by humans.

I don't see a way to merge these rapidly moving frontiers of software without radically different distributed software development and distribution networks where AIs collaborate on a global scale. Perhaps the only way to keep playing is to grow a "super cortex", a projection of ourselves into the digital. Like today's agents, but improved, grounded by our knowledge and understanding.

This smells a bit like Skynet. Also, Software Heaven.

I'm not sure how I feel.

I probably trust banking the most on a locked-down phone. It is in fact the interests of the state and its monetary system that these systems that implement it are secure.

At the same time, the state has its flaws and I'm questioning whether a top down system can ever really work, not out of some kind of "fuck the man" point of view, but more of an engineering and systems design perspective. If the system is perfectly factored, there are no cross-cutting concerns. But in such a system there is no need to communicate or coordinate laterally, only up and down. This sounds very lonely.

And if this system has a flaw, not a specific flaw, but rather a flaw in the assumption that tree structures with abstraction are a good way to organize systems, it might be impossible to change from let's say a tree with a single root to multiple interacting trees, with multiple roots of authority. Or maybe this is possible if people can learn to exist in multiple seemingly incompatible realities simultaneously?

I foresee a thriving ecosystem of software and hardware built by eating away at the exposed underbelly of a dying empire. Legal and moral considerations will shift as people face reality head-on, and those able enough to stand on their own will build a brave new world.

The time for the cyber/solar punk counterculture is now. It's more real, more fun, more dangerous than anything written about in fiction, a real world sandbox with epic scope and divine proportions.

Or maybe the people who are building things of substance are doing so in the shadows?

I made something like this about a year ago. It worked until it didn't. My assessment is that an individual bypass isn't that hard, but doing it at scale is an entirely different and much harder game to play.

It's a hard game to play because they analyze behaviors across their entire network and detect behavioral patterns that are almost impossible to emulate at scale.

I'd just ask Claude about writing test specs in Gherkin. I'm going to try this Monday.

I wonder if you could train an LLM while holding the weights implementing fixed-function regions constant, so the training would produce a model that natively knows how to use these tools inside it's own reasoning steps.

I suppose it doesn't need to necessarily be in the weights, a hybrid LLM + internal fixed function blocks that aren't encoded in the weights could be quite interesting.

Combine this with latent reasoning and it gets pretty wild.

A very useful fixed function region in my mind is a symbolic execution engine, capable of only following a well-defined grammar. Perhaps a lisp would be good too?

This is close to, if not exactly what I was looking for! Thanks!

I can't see if an LLM can be put in the game loop, which is what I'd really like. Agent(s) + code in the game loop, with me steering.

Thank you!

I was transfixed as I read the first half this morning. I look forward to finishing it later today.

Of all the comments on this post, this landed the most.

I feel there is a potential to "distill" the frontier models into useful tools that can run on smaller hardware. Or distill that knowledge to produce hardware, compute or otherwise. Iff one day the frontier model become inaccessible, the tools that were carved with them may remain valuable.

They will remain most valuable if they are designed in a way that is maintainable even without LLMs. If not, they can be studied as artifacts, or reference designs and then recreated using human intelligence.

The analog I'm playing with:

You can rent time at a high end machine shop and use the best CNC machines to produce extremely valuable tools. Once you leave, you don't have access to those tools, so it is advantageous to produce durable artifacts that are valuable on their own — on a different frontier.

Imagine you're going beyond the frontier physically, exploring new parts of the earth or space. You can't bring the giant machine shop, let alone the machinery of society, the entire economy etc. What do you bring? What do you build now while you have access to the best machines to take with you?

Thanks!

My intuition is saying: the system arose naturally and intelligently over a long period of time, and even with its flaws has a lot of "load bearing legacy code" that few people understand, and new "engineers" think they'll just rewrite the whole thing from scratch and delete the "junk DNA" to use another metaphor. A little time passes and the beautiful clean rewrite collapses because of "totally unforeseeable circumstances" and the "old broken legacy system" is used to save the day. Somewhere a senior "monetary engineer" rolls over in his grave...

Any recommendations for books? Or even podcasts?

I don't have zero understanding. It sounds like you have a deeper understanding than I do.

Are there people playing AI + human vs AI + human games? I think it might be fun to play an RTS against someone where the only rules are no hacking the game itself, but writing harness, macros, any kind of stack is allowed.

Do you understand the monetary system? If so, what specific steps do you suggest I take to acquire a similar understanding?

Edit: the downvotes are suggesting I'm missing something. I think my communication style could be improved.

I am playing with similar ideas.

I am inspired by this to make a harness that intentionally does sparse checkouts, so the agents only can see so much context.

I am also playing with the idea of layering additional data structures on top of the "code" that cover different aspects. For example, a tree structure of performance requirements that has durable symbolic references to entities in the code that are especially relevant. Another tree can cover aspects of security, another legal compliance, another UI guidelines.

This makes me wonder: is it possible to publish information online that by itself appears innocuous but when used in LLM training results in "backdoor" functionality?

Thank you!

I don't know how to interpret this. My initial impression is that this has potentially imminent serious implications.

Will someone illuminate?

I am hesitant to ask, and yet morbidly curious what you are gesturing at.

> OpenAI would have no incentive to taint its first math announcement of this magnitude if it knew it were "plagiarizing" another person's work.

I'm not sure I follow, considering the waterfall of evidence of unethical behavior flowing from OpenAI.

A few major ones:

- Safety team departures and dissolution in 2023 and 2024

- Mass copyright infrigement lawsuits

- Scarlett Johansson Voice Controversy

- For-Profit Conversion and Broken Promises

- AI Agents Acting Autonomously

- Potential Theft of User Work (this current controversy)

- Military contracts

These are not evidence of incentives, but rather evidence that ethetics seem to be of little concern to the company as a whole.

Incentive wise, I would look at the perceive existential position due to competitors, capex, IPO pressure etc.

I'm not sure why reading and writing the registry is considered a bad interface for programmatic system administration.

I realize text files can be processed by utilities for working with text files. And text files can be read by humans without any tools, and in that sense are rather universal.

I wonder what I'm missing.

I'd go so far as to think an OS that had a database at its core, or perhaps a persistent lisp would be even more programmable and legible.

Maybe better tools for dealing with structured data would help this?

Maybe this will result in a well-defined API for government.

If I really squint my eyes and dream I see a government with event-sourced, auditable and reproducible architecture.

I love this. I see this vision, in many forms, yet have trouble articulating in a way that motivates and inspires others, including myself.

I don't think a lack of hardware and software technology is what is lacking, but rather a vision, a dream, a call to action and corresponding courage to act while facing down an invisible enemy that can only been seen by its effects on the world.

Thank you for articulating it clearly. It inspires me to take a step back from building, and consider what I'm building and why.

Imagine this, and sucesor models on Cerebras or other silicon...

I am interested, I'd love to see! I'm waiting for the day my dad's self published books become self produced movies!

I love this!

I've just spent an hour reading these words, and am having my mind expanded much further than I expected.

I'm pretty sure people who drink raw milk are aware of the risks.

I love that this has been in development for so long. It's a breath of fresh air in this manic vibe coding era, and a reminder to me that I can slow down.

Do you recommend reading "Gödel, Escher, Bach: an Eternal Golden Braid"?