Hacker News Reader

Comment by verdverm | Hacker News Reader

someone replied with this meme and said, "wait until Jev learns to drift"

https://www.reddit.com/r/trolleyproblem/comments/15npmsg/mul...

I've been using coin flips to demonstrate basic failings for decision models, i.e. these are not real probabilities, or they depend on the training data, which may not have an accurate sample to the true population value. We need to collect more failure modes for all Ai so that humans can better judge when and how to use them safely.

View on HN

haha, I'm afraid to offer the models that ability. Where did they reply? The model labs are sitting on a trove of data on failure modes for AI, but I bet you they aren't sharing it with each other and certainly not with us. It makes them look bad. My speculation, of course. I don't work in alignment research.

I dropped it in our slack ai chatter channel

The bots will find new failure modes we did not think of, clever workarounds to anything, because we don't give them the right context and instructions

well, it's a nice property of decision models that they can only choose from the options you give them. As I write in the article, I think that's a way in which they are safer than LLMs

I would contend that you can enforce the same determinism with some plumbing and that the thinking process is more helpful. My read is that people see the response time for decisions as one of the bigger values, useful in tasks like content moderation, which doesn't carry the same risks as life-death decisions.

I'm working on an NES harness to play Final Fantasy 1 to demonstrate this, as compared to Jev playing Pokemon. Did you happen to catch that while it was live? (not recorded, but it shouldn't be hard to replicate)

The thinking process is helpful, but it's not strictly predictive of an LLMs output, and its not really the model's "thinking", although it clearly has an effect and a benefit. I cited one paper on the topic, Turpin et al. So I'd be wary of reading the thinking tokens and inferring they tell you how a model decided something. And an LLM can output things you never asked for, including dangerous things. With a decision model, you can just constrain its choices to those you're comfortable with it making. An LLM requires another layer of interpretation of its outputs to get the same property.

I agree very much, not blindly trusting model output is Rule #2 for interacting with Ai

I use the LLM coin flip as a comparison point. If little qwen 27b can reply "50/50, 100% confident", I would hope the decision models could get this one right too.

Reply on news.ycombinator.com