104 karma · joined September 3, 2024
If we're talking about simply cutting costs, sure -- but those savings will typically be reinvested in more talent at a growing company. Then the bottleneck is how to scale managing all of it.
It would be neat to also see another experiment of a MAS doing this and coordinating to gamble together. Perhaps even different system/arch/expert configs.
Maybe it's like:
1. Intention, context 2. Attention scanning for components 3. Attention network discovery 4. Rescan for missing components 5. If no relevant context exists or found 6. Learned parameters are initially greedy 7. Storage of parameters gets reduced over time by other contributors
I guess this relies on there being the tough parts: induction, deduction, abductive reasoning.
Can we fake reasoning to test hypothesis that alter the weights of whatever model we use for reasoning?
I've first thought of this within the concept of self-assembling autonomous agents in 2016. Good times dreaming about a future where AI permeates every facet our lives.
I don't really care to memorize (which was most of the coursework) things which I can just easily look up. Maybe geography in the south was different than how it was taught elsewhere though.
When I was young, I refused to learn geography because we had map applications. I could just look it up. I did the same for anything I could, offload the cognitive overhead to something better -- I think this is something we all do consciously or not.
That attitude seems to be the case for students now, "Why do I need to do this when an LLM can just do it better?"
This led us to the conclusion:
1. How do you construct challenges that AI can't solve? 2. What skills will humans need next?
We talked about "critical thinking", "creative problem solving", and "comprehension of complex systems" as the next step, but even when discussing this, how long will it be until more models or workflows catch up?
I think this should lead to a fundamental shift in how we work WITH AI in every facet of education. How can a human be a facilitator and shepherd of the workflows in such a way that can complement the model and grow the human?
I also think there should be more education around basic models and how they work as an introductory course to students of all ages, specifically around the trustworthiness of output from these models.
We'll need to rethink education and what we really desire from humans to figure out how this makes sense in the face of traditional rituals of education.