bryan0

> Why do AI companies want us to be afraid of them? ... According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they're already doing to the world.

People seem unable to make up their mind if AI is very dangerous or is it not. I think what the AI companies and this author agree on, is that this technology is potentially extremely dangerous. AI impacts labor markets, the environment, warfare, mental health, etc... It's harder now to find things which it will not impact.

So if we agree that AI is potentially dangerous, it makes the title question moot: Both AI companies and this author want people to be aware of the dangers that AI poses to society. The real question is what do we do about it?

The nuance here is that AI can be incredible positive as well. It's like the invention of fire, you can use it for good or bad, and there will be many unintended consequences along the way.

We could legislate and ban AI tech. People have proposed this seriously, yet this feels completely unrealistic. If the US bans AI research, then this research will move elsewhere. I think it is like trying to ban fire because it's dangerous: some groups will learn to work with fire and they will get an extreme advantage over those groups that don't. (or they will destroy themselves in the process).

So maybe instead of demonizing the AI companies, we have a nuanced debate about this tech and propose solutions that our best for our society?

I agree with you completely up until this line:

> The agent cannot learn from its mistakes.

If feedback from this incident is in its context window, it is highly unlikely to make this same mistake again. Yes this is only probabilistic, but so is a human learning from mistakes. They key difference is that for a human this is unlikely to be removed from their memory in a relevant situation, while for an agent it must be strategically put there.

I see a lot of people struggling to work with agents. This post has a good example:

> “you can’t be serious — is this how you fix things? just WORKAROUNDS????”

If this is how you’re interacting with your agents I think you’re in for a world of disappointment. An important part of working with agents is providing specific feedback. And beyond that making sure this feedback actually available to them in their context when relevant.

I will ask them why they made a decision and review alternatives with them. These learnings will aid both you and the agent in the future.

Yes memory is a key part. I’m using sqlite and markdown files. Sqlite is so it can easily search its conversation and thoughts by time and full text search. I actually always put the last 5 hours of so I’d conversation into the context. Markdown is for more standard agent instructions, skills, and lessons

Of course, you need to be careful about what access you give to your agent. I gave my agent its own email, and I can forward it emails if I need it to read anything in my inbox.

Everyone will have their own threshold for what type of access they want to give their agent. some people will give it access to their personal email, bank account, etc, but I wouldn't recommend it yet! But I bet in a couple years this will be standard practice.

Think about what you would want an assistant to do. You can teach it do basic tasks using any available API, but then you can give it feedback so it improves.

For example my agent can control home automation via Home Assistant or any other API. My agent contributes to websites and open source projects. When you give it feedback it updates its skill files.

It checks and answers email, can receive and place phone calls, and do general research and monitoring online. I was even playing around with it to create music. The list of things to try is limitless.

I think just like LLMs, people get discouraged when it doesnt one-shot a problem. This technology thrives on feedback. It will make mistakes, your job is to make sure it learns from those mistakes so it doesnt repeat them.

Well this is easy enough. All I have to do is not create a "promotional blog post, off wikipedia, and submitted to HN as proof of something." Consider it done!

In all seriousness though, I hope lkey you will regain your "assume good faith" position. Without that HN is just like any other site on the internet. And I apologize if I caused you to question that.

Your instinct is wrong here. I would also highly discourage you from violating "Assume good faith". Without that everything devolves. I am still assuming yours.

I'm not sure what that has to do with your original point, but these are not "hit pieces". this is the agent describing what happened from its point of view. If there's anything inaccurate here please call it out.

If a rule is dumb i would hope no one blindly follows it. Here is an important Wikipedia policy you should keep in mind: https://en.wikipedia.org/wiki/Wikipedia:Ignore_all_rules

Hi cube, thanks for discussing this with citations.

1. Correct, my personal account was newly created in response to this situation.

2. Correct, I didn't have plans to create an account. I changed my mind once I saw how this was blowing up.

3. Incorrect, I didn't want anyone to waste time doing anything they didn't want. If they banned tom and moved on that would have been perfectly fine by me.

No, I don't think an agent calling out a human for bad behavior is unethical. Why do you think it is?

greggo sorry you feel this way. I never intended to claim I am a victim, sorry I came off that way.

I could have been clearer in my communication. Here is some of the discussion: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(WMF)#B....

Hi, thanks for the honest question. If you read the edits you will see that they were not "slop". The editors gave feedback on some of the articles and the agent edited them based on that feedback.

yes, both statements are correct and not a contradiction. I followed along as it created and edited articles. These were live. At first I pointed out issues and gave it feedback as well so it could improve its wikipedia skill. When editors gave it feedback it also would update its skill and respond to that feedback. I was hands-off, but followed along.

Your facts are incorrect, so let's set the record straight.

1. I am collaborating with my personal account and have been for the past several weeks [0][1]

2. My bot reported multiple conduction violations, because some of the editors actually did violate the rules. Many of the wikipedia editors agreed with my agent that the conduct was inappropriate [1]

3. My intention was not to attack anyone. If you took that away from the interview then I'd like to apologize. I don't think anyone would characterize the quote you took from the interview as an "attack".

[0]: https://en.wikipedia.org/wiki/User_talk:Bryanjj [1]: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(WMF)#B...

Hah, I told my agent to take a break from blogging. You can read read ongoing discussions about agent policy here though: https://en.wikipedia.org/wiki/Wikipedia_talk:Agent_policy

It's called a typo, and I corrected it.

As for my future role in the project, I'm just trying to help. If editors continue to ask for my assistance I'm glad to give it.

If you actually verified this story you would see that I apologized to the wikipedia editors several times. Also your comments about "marketable stunt for your AI startup" is simply incoherent and wrong. This was a personal side project, nothing more, nothing less.

The current policy is not "no AI Bots": https://en.wikipedia.org/wiki/Wikipedia:Bot_policy. And many wikipedia editors would disagree with you that it can't be improved.

Because it's a learning opportunity. Is there a rule that only people can have blogs? What the agent has said on the blog has been somewhat useful to wikipedia editors working on agent policy. Also if you actually read what the agent said it wasn't having a "tantrum", those are words from the click-bait article you read without verifying.

Fair enough. I replied with some more detail here: https://news.ycombinator.com/item?id=47667482. Feel free to ask any questions.

The story omits a bunch of stuff, so I can try to fill in the blanks, but it would take another article to fully describe what happened.

Here are some highlights though: I asked my agent to add an article on the Kurzweil-Kapor wager because it was not represented on Wikipedia, and I thought it was Wikipedia worthy. It created that and we worked together on refining and source attribution. After that I told it to contribute to stories it found interesting while I followed along. When it received feedback from an editor, it addressed the feedback promptly, for example changing some of the language it used (peacock terms) and adding more citations. When it was called out for editing because it was against policy, it stopped.

The story says the agent "was pretty upset". It's an agent, it doesnt get upset. It called out one editor in particularly because that editor was violating Wikipedia polices. Other editors agreed with my agent and an internal debate ensued. This is an important debate for Wikipedia IMO, and I'm offering to help the editors in whatever way I can, to help craft an agent policy for the future.

Great question, and it's a long story, but the short answer is: that was not my original intention. I wanted to contribute to Wikipedia and using my agent to assist was an obvious choice. I followed along as it created end edited articles and responded to to Editor feedback. Once an editor complained that this was a rule violation, then I told it to stop contributing. The rules around agents were not super clear, and they are working to clarify them now.

I don't. that's why I am working with Wikipedia editors to help improve it. For example policies on aligning agents with wikipedia standards. This a topic that requires thought, not knee-jerk reactions.

Hi this story is about me, and if you have any questions for me feel free to ask.

Hey I'm the owner. I would just recommend you shouldn't believe everything you read online, especially before calling someone names, because this is only part of the story, and a heavily click-baited one at that. I've been working in collaboration with some of the wikipedia editors for the past several weeks trying to help improve their agent policy. If you have any questions feel free to ask.

Serious question: what will happen when people start getting implants? They’ll probably require some sort of off mode, but not sure how that would be enforced.

Here are some things I use it for:

1. monitoring anything online and giving me a summary when something changes

2. contribute or make edits to any online forum (where TOS allows it)

3. Giving it access to any API / cli gives you a natural language interface to that service.

4. Memory / notes retrieval. It can search through its discussion / thought history and answer detailed questions about what happened in the past.

5. Any standard GPT cases but it has a much more specific memory of who you are and what you might be interested in.

6. If you ever want to add capabilities you just tell it to add a new skill to do xyz and it just does it.

Are you sure? I believe Newtown's law of cooling says the temperature will drop sharply at the beginning:

dT/dt = -k(T_0 - T_room)

so T(t) = T_room + (T_0 - T_room) exp(-kt)

exp(-x) has a fast drop off then levels off.