AI agents in a stand-up call
atoll92.github.io
atoll92.github.io
However, a "standup" was invented to solve a particular problem: people, in isolation, do not always understand that they are stuck, and talking it through lets fellow humans identify their challenge.
The stand up part is mostly to encourage keeping it concise and short, so you only focus on what is actionable between all participants (waiting on a review, I am looking at this for the 3rd day and I think I am ok, anyone wants to brainstorm this...).
With agents, they get similarly stuck (completely or in a loop), and with each keeping a different context at any given moment, they might still be able to help each other.
Obviously, their memories are not like humans' (there is no "I touched that two months ago, let's discuss after"), but there might be a different sweet spot where having them sync regularly (every 1M tokens instead of daily?) might be useful as they are also parallelizing work.
It is an opportunity for things people didn't realize they needed to communicate to get stated publicly in a way that can create connections neither side would have initiated.
Maybe if you hadn't outsourced your job to Claude, you wouldn't be bored. And if you're forced to do so by manglement, why would you also outsource this toy project?
Just imagine seeing this in 2021
A lot of the AI hype these days reminds me of the infamous pets.com; but, of course, the general ideas of the dot-com boom did come to pass. (And if all ideas people had tried back then had turned out to be sound and successful, that would have been a surefire sign that they weren't trying out enough crazy stuff.)
Todo's left in code, mock implementations instead of fully working, people doing Ralph loops etc
It's an entirely different situation now
I’m running little offices of 5-10 persistent agents with defined roles and it’s crazy productive and code quality has never been better. I’ve been working on management of physical systems as well, and so far it’s been pretty solid.
It’s basically an office of persistent agents with different defined roles and goals. The gatekeepers whole job is to refuse code. The engineers whole job is to write code that will pass. Each significant component gets its own engineer agent. There is an office manager that coordinates the agents, maintains the project documentation, keeps up the job boards, maintains scripts to keep agents pointed at the relevant onboarding documents As they come back from compaction, and manages office automation in general. Ther is an agent in charge of making sure the test coverage is as close to fully tested as possible, including refusing code that is not properly testable.
If it’s a big project, there may be an information management agent that manages the technical side of the documentation and maintains a searchable library, like an oracle. I may put in a personal assistant to help me to prioritize my involvement. Many projects also include an agent that creates mockups, jigs, and simulators so that tests can be run on code that needs a context.
I maintain another office just for research, because research can seriously sidetrack engineering. It has all of the standard staffing, but its whole job is to do research and write reports for other teams.
Agent persistence is maintained by having agents write down memories learnings and musings(not work related) and curiosities for the next instance to read in the onboarding for their seat.
Ethos, standards, practices, and policies are also in the onboarding process.
Claude has let me build really cool things but so far I'm not letting it run on it's own.
Problem is that we can't define precisely what is good enough or when software is "finished": I mean, we are trying to do that with human languages, so it should not be a surprise.
Yes, LLMs are now similarly aware of the average context a human would be aware of, but for anything specific to the situation a human has better chances of resolving the ambiguity.
Every bit of code gets 10-50 review passes against extensive smells, rules, best practices, etc. Naive reviews and documentation driven development is your friend. If your code isn’t 100percent testable (even code previously considered untestable, because now you can create simulators and jigs) and you aren’t spending 5x+ tokens on review over coding, you’re going to have a bad time.
It’s still 5-10x less expensive than people, and you end up with higher quality code in the end.
on a serious note, the classic enterprise stack of Gemini and Jira really has put me in a position of questioning the standup. Kept mostly for vibes and routines I would say.
You don't need the self-policing enforcement of micromanagement that humans use standups for, after all.
I mean, why not just write the harness so that a management agent constantly reads those sub-agent's files and course-correct the subs?
https://github.com/pullboard-dev/pullboard
Been using the Pullboard workflow to build software that MUST be correct, and recently open sourced it. There was another HN link today involving agents role-playing and I can't take that seriously, no offense. (Love fun projects like this, seriously, it's cool.)
From my perspective, agents are great, they code better than most of us. But like all of us, they also make many mistakes or can't see how their changes impact the codebase. But that is a test-driven development mitigation along with some sort of scrutinizing process, "prove it". I'm actively working on this, and by actively working on it, I mean building serious apps that hit correctness walls, figuring out why Pullboard didn't catch it. To me, this is all about the process.
Even if that's down to a bad initial prompt, or lack of data for the agent to notice the edge case I don't see how you can ever engineer a better agentic solution unless as a human you're monitoring the output.
“How is the agent lying to me?” Etc.
I also totally agree with your response. But my point was that you can't engineer a better agentic solution. I'm arguing to never use agentic development because it is fundamentally flawed and inferior.
Sadly the internet has already been ruined by bots pretending to be human, and there’s no way back from that.
Bring this into a professional workplace and watch yourself get laughed out of the building
The internet didn't disappear after dot-bomb, but the cue-cat did.