321 karma · joined August 7, 2011
If that could be done, open source maintainers might be able to effectively get free labor to continue to support open source while members of the community pay for the tokens to get that work done.
Would be interested to see if such an experiment could work. If so, it turns from being prompt injection to just being better instructions for contributors, human or AI.
This makes me think of risk assessment in general. There's a tradeoff between risk and reward. More risk might mean more _potential_, but it's more potential for both benefit and ruin.
Do you think we'll figure out a good balance?
This is referring to the software the agent is working on, not the agent.
> This creates a continuous feedback loop.
This is referring to the feedback loop of the agent effectively compressing learnings from a previous chat session into documentation it can use to more effectively bootstrap future sessions, or sub-agents. This isn't about altering the agent, but instead about creating a feedback loop between the agent and the software it's working on to improve the ability for the agent to take on the next task, or delegate a sub-task to a sub-agent.
> "... the type of self-improvement we’re talking about is far more pragmatic and much less dangerous."
This is a statement about the agent playing a part in maintaining not just the code, but other artifacts around the code. Not about the agent self-improving, nor the agent altering itself.
I think this is a common, but incorrect assumption. What military commanders want (and what CEOs want, and what users want), is control and assistance. They don't want a system that can't be turned off if it means losing control.
It's a mistake to assume that people want an immortal force. I haven't met anyone who wants that (okay, that's decidedly anecdotal), and I haven't seen anyone online say, "We want an all-powerful, immortal system that we cannot control." Who are the people asking for this?
> ... it will do whatever it can to prevent it being turned off.
This statement pre-supposes that there's an existing sense of self-will or self-preservation in the systems. Beyond LLMs creating scary-looking text, I don't see evidence that current systems have any sense of will or a survival instinct.
> ... if the docs act like a summary of current state, you can just read it at the start and update it at the end of a session
Yeah, exactly. The documentation is effectively a compressed version of the code, saving agent context for a good cross-section of (a) the big picture, and (b) the details needed to implement a given change to the system.
Think we're all on the same page here, but maybe framing it differently.
Outside of AI, the combination of "a given actor can make their own decisions, and they have unlimited permissions/access -- what could possibly go wrong?" very predictable bad things happen.
Whether the actor in this case is a bot of a human, the permissions are the problem, not the actor, IMO.
Inspired by all of these examples, but desiring a dead-simple solution for the Ruby applications I maintain - I created nobject.
It's not quite RPC. It's not quite the Actor model. It's not quite lightweight processes/channels. It's the ability to instantiate an object in one process but then push it to another process, yet be able to use that remote object like it's still local.
The example code in the repo's README will get you up-an-running in minutes.
While the author says that they are bypassing Google, that's not most people, and Google's results are front-loaded with AI answers, so Google results are already giving you specific answers, hallucinations and all. Not sure why average users would long-term switch to not-Google if Google can give them the amount of AI assistance that want or don't want.
Ex.: People were writing about agile, no-code, and the challenges of reducing cost and complexity before we had the terminology for it and before a whole industry of consultants existed to explain it.
Application Development Without Programmers https://a.co/d/2kKeOTx
Most people are taking this as a fun algorithmic exercise, assuming imperfect knowledge.
The last half sounds a little bit too much like the bias towards markets and profit motives and whatnot without question and without critique. There are plenty of dysfunctional for-profit orgs that don't allocate resources well, in spite of overwhelming feedback, from the market and elsewhere. The majority of for-profit orgs also have a sales team that is out there pounding the pavement trying to drive sales: it's not all, "Build it and they will come."
I don't buy that a successful company needs to be a mission either. That's very common thinking in the startup world, but not even close to representative of the whole.
None of the above means that I don't see MySQL as flawed in some ways. I'm in a group of developers that I suspect make up a sizable portion of the MySQL community who didn't choose MySQL, but must support it, if for no other reason than because we see ourselves as professionals, and that's what professional do: make the employer's application work reliably.
For applications that have already survived past the point of finding product/market fit, a wholesale conversion of DBMS is rarely worth it, and conversions of this type are costly/risky even if it is worth it. I do understand many of the benefits (real and theoretical) of PostgreSQL, and if I'm around at the moment when a project's DBMS is being selected I'm going to recommend _not_ MySQL, but at some level I'm also paid to make the application that my employer is running on top of their DBMS work reliably ... and the fact is even among people who get PostgreSQL - who prefer it, would choose it if they could - many of us are also pragmatic enough not to pull the rug out from under a running application for "reasons".
For a broader view, years ago I read the book, "Power: Why Some People Have It—and Others Don't" by Jeffery Pfeffer, and it was an excellent object lesson in understanding how and why companies work the way they work, how decisions are made, and why you can do the same job in different organizations and yet have a totally different experience one organization to the next.
There's research that's talked about in the book, "Data and Goliath" that explains how the behavior of people specifically trying to avoid being tracked is sufficiently different from most other people that, even if you turn off your phone (sometimes ESPECIALLY if you turn off your phone) that act can be a behavioral marker used to correlate your activities with other people who do similarly, and your location can be largely deduced by process of elimination anyway.
It's a fascinating read.
I mean c'mon - have a little fun.
At any rate it's classic absurdist marketing, and I approve.
Besides, I don't read the TOS of most other services. Why would I start now?
If you think of electricity as fuel for computers, similar to fuel for a semi, then this scheme seems analogous to an attempt to get truck drivers to drive your stuff to it's destination at their upfront cost in exchange for an IOU that has no backing.