2,711 karma · joined March 20, 2015
Dr. GPT is a good brainstorming tool. It helps synthesize information in a way that primary texts don’t. But it does force you to say “that doesn’t make sense”.
I do think that people saying “doctors don’t know the state of the art” have a weaker case. If you think about it in terms of token density during pretraining and how post training datasets are constructed, I think it would take us a very long time to adapt to any fundamental shifts. If we have forgotten how to cure scurvy, how many journal articles would it take before we adapt to a discovery?
What I’ve seen be the true bottleneck is people not setting up the structured data. But making a tiny reasoning model with OPSD -> GRPO is totally doable with a bit of money.
I think we’ll see a lot of specialized VLMs that provide real value.
There is a huge contingent of people who do not interact with AI on a daily basis. Many of them legitimately believe we're seconds away from wiping away white collar America. Many more believe that we are creating the literal singularity.
None of them have seen claude try to model a problem they're familiar with.
I was working on creating a next-n-actions predictor for one of our use cases and not paying much attention for a PoC. I was fairly happy with the progress for a few days, before actually reading the eval code and seeing that we leaked the final state in every eval.
It's nice to let claude run loose on porting from framework to framework (port my code from TRL to NemoRL to Tinker to VeRL) but looking at what it does in the intermediate steps makes me want to claw my eyes out. And getting it to adhere to our domain model (e.g. we have an SFTConfig and a .to_trl(), or a Row and a .to_harmony()) is impossible.
Imposing a structure on this unstructured discovery feels like it ignores all progress in IR, and imposes a farcical structure that doesn’t have a tangible benefit.
Both were trivial to set up with codex.
“which plays on this general tendency of humans to prefer comfort over challenge, confirmation over rejection.”
This is completely and observably wrong. It reads like it would make sense, but the most ideologically open cultures I’ve seen are LIT.
One implication was people were more social and talked about ideas more. Thought had not been outsourced to arbiters in the way that it was in the U.S. People with authority, knowledge, and close family members were definitely inputs into what people thought, but by-and-large people still came to their own conclusions.
You got to see the gradient of thought that people actually had about issues. People would say their insane ideas out loud. You could disagree with people and have them actually engage with your perception of reality.
It was strictly better in my opinion.
Moving there, I was shocked at how "conspiratorial" everybody seemed about everything. Living there, I was shocked out how often they were right. But it didn't impact people's likelihood to do things. I think they are actually orthogonal in a way that is unintuitive.
Wishing you luck with the project!
Declarative approaches require validation to live at a synthesis layer, while an imperative approach that compiles down to declarative configs at runtime gives you the best of both worlds -- this is why anyone who does not need terraform cross compatibility will write things against CDK or Pulumi that has the same declarative schema wins with the niceness of testability and author-time typing.
Edit:
That said, it is shockingly close to the schema that we wound up with with a few ideas that I think are interesting.
reportsTo allows bottoms up orchestrator delegation
workspaces are interesting -- right now we have one bag of data with per-subagent data subscriptions, but this means that we frequently add input requirements to subagents that really should be more implicit
accessPolicy seems like a footgun to me -- i feel fairly convicted that tools should define their access scope and the only thing a subagent should know is the bag of tools available to it.
human approval seems redundant given we already have input requirements, and one can just be `email_approved` with a tool that emits the human approval request and `email_approved | email_not_approved` -- same feeling about `GateTypes` in general. If we are working on flat input-output requirements, then why do we need a specific GateType handler?
Trigger `any_approved | all_approved` is going to bite you if you move into plan solving. It is not rich enough to express XOR style relationships and I am willing to bet that v2 of your implementation splits TriggerRequirements where TriggerRequirements can be recursively applied to the type.
It seems like the tool definition is missing a lot of niceties that have been important for us -- for instance, at most once invocation. But we are working primarily over voice where there is a strong need to control execution for quality of service.
The tools we need to solve this problem exist and they are boring. Types, jsonschema, openapi, all of it is a better integration point than MCP.
We started in Python because of “the ecosystem”. It was a mistake. The amount of time we spend ripping out each dependency and pruning it to what we need is way higher than if we’d spent the month building out what we need. I miss compilers and LLMs will NOT generate config driven code or things that serde well. Everything has layers and layers of adapters by default and the domain model slowly erodes over time.