vkgcr

I've been working on Semiotique (still very early stage).

Right now it's a python library, but the vision is to build a new framework / SDK / programming language which allows you to seamlessly mesh "plain old code" with LLM evaluations, while having a runtime which does "smart" context engineering in the background in order to steer the LLM and improve its correctness.

The end-goal is for this paradigm to enable new kinds of agentic applications, which leverage the flexibility of LLMs while retaining the determinism of regular code.

https://github.com/VKCR/semiotique

This sounds like a pretty dumb explanation due to the obvious fact that you're not forced to only read books which came out in the current year...

Completely agree.

It's honestly surprising how little work gets put into building harnesses which carefully steer the LLM through sophisticated context curation. Probably due to the belief that we shouldn't care much because AGI is just around the corner.

Related (but shameless) plug : I've actually been hacking a project over the past two weeks to address this very issue. I'd like to create an SDK which embeds the evaluation of LLMs into code seamlessly, and does "behind the curtains" context curation, to really enable "hybrid" code/LLM agentic applications. (if you're curious => https://github.com/VKCR/semiotique)

Do you also believe the same for other categories of products? For example, for a car, or for a building? Would you board a plane built by people who focus on business value at the expense of the annoying "high standards"?

Product quality is an emergent property of high standards (aka code quality).

I know you're being sarcastic, but I meant prompting an LLM with "generate me a X billion parameter model for doing Y task" and it would just output the weights.

Now THAT would be crazy

But theoretically, what would prevent the labs from RL'ing the models on "weight-space programming" tasks so that they get better at them? Perhaps we're just at the beginning, like how coding agents sucked 2 years ago.

It still seems like a promising approach for RSI, distillation, fine-tuning, and a whole host of other tasks.

It even seems to me that plateau-ing is an inevitability. Unless there is some way to reach AGI which you could deduce from a combination of ideas which already exist in the training dataset, but that seems rather unlikely.

This opens up a whole new world of possibilities. Producing a custom fine-tuned model from a prompt...

You can be bullish on LLMs profoundly transforming society while being bearish on the economy "digesting" this new technology fast enough to justify the incredible capital spending necessary to power it.

You bring up some interesting points. I certainly think that you can go overboard in the other direction, like in South Korea, where I'm not sure the crazy intensity of their educational system is necessarily better.

I still think there would be benefits to making education in the west a bit harder, if for anything to build the taste for intellectual effort in young kids. But especially to identify talent who could not be identified otherwise.

It's interesting that you mentioned Tao and Carlsen. Would they have been identified by the educational system had Tao's mom not been a mathematician, and Carlsen's dad a chess enthusiast?

More controversially, assuming "normal" parents, would Carlsen and Tao have had more chance of being identified and pushed towards excellence had they been born in the USSR in the 1950s or in the west today?

Oh yeah for sure, but I think that's the case "in spite of" the trajectory of the educational system.

From my personal experience, people who are successful usually come from highly educated and/or wealthy families. I don't think school plays that much of a role in the grand scheme of things compared to family influences. There's a lot of inertia to society in general.

However, what we lost is that modern educational systems are really bad at identifying and "pulling up" talented kids from poor / under-educated families, because the standards and expectations are so low that you can't differentiate them from their peers.

I assume because they had not adopted the "dumb everything down to reduce variance in outcomes" educational ideology that has taken hold in most of the west over the past 50 years.

I interpreted the parent as claiming that redistributing wealth would see the software industry's product offering driven by the "working class" rather than decided by the CEOs of corporations.

I don't disagree that it was better / less soul-sucking in the past, but I think that's because there was a lot less money in it, so participants in the software industry were in it because they were passionate. Unfortunately, corporatization is necessary to scale.

I'm sure you can find industries today which are largely driven by passionate people working in small outfits (but there's probably not much money in it).

All of the examples in the article are products from startups or corporations, though. I fail to see how more wealth redistribution would improve product quality.

I think this is an uncharitable take on PG's tweet. I interpret it as acknowledging the limits of AI.

I think the more interesting conversation is : why are AI's outputs still so "sloppy", whether we're talking about text, images, or even code? And if it's due to inherent limitations with LLMs, then it should make one optimistic about the future!

I'd be curious to know how the "weights are conscious" crowd interpret this model!

But wouldn't that be horribly inefficient? Instead of executing code by transforming it into a set of weights and running inference on them, wouldn't it be easier to just execute code as... code?

It's been one of the most eye-opening things for me as well. People who I considered serious and intelligent completely falling prey to "no need to read the code" AI psychosis.

Perhaps it reveals something deeper about about differing personality traits, about people who like programming as a discipline and like understanding code, vs people who view at as a means to an end.