Hacker News Reader

Comment by luis_valerio | Hacker News Reader

To be precise the thing provide permission to do any operation, including destructive actions to the agent but we do set policies, auth controls and hitl force approval to validate the destructive or high impact actions before apply them

Really good point about using the same context for the compliance, currently it's a separate subagent but read rules/policies and then check against the other agent proposal, no possible to inject since we control totally via a rail the prompt for the compliance subagent but it's true that it checks policies so it's not deterministic (works pretty well but not deterministic) so that's we are targeting checking that policies deterministically with OPA and then the compliance will only add to the final result some other checks that need to be LLM inferences.

So it's pretty similar to yours, you have any other additional harness to be as much determistic as possible?

View on HN

Yeah, sounds very close. The main thing we did to keep it deterministic is that nothing at request time is a model call. Rules are evaluated in the proxy against the actual request: method and path, the provider operation it maps to (we classify each provider's API into operations and families like delete or payment, from their published spec), parameter values, IP, time window, quotas. Default deny, and every rule can only narrow.

The other half is treating policy like code. Rules live in a file with test fixtures ("this request should be denied, this one should need approval"), `alter policy test` runs them, and `plan` / `apply` works like Terraform: you review a diff, and apply refuses if the plan was edited or the live policy changed since you reviewed it. `alter policy simulate` shows which rule fired for a given request and why.

Understood yeap. Yes exactly is terraform/opentofu. The point is that we cannot do it at proxy level since we do allow any kind of operation attempt (we just filter that it's an infra req), then we generate the code based on the policy and, as you mention, once we have a validated plan we crosscheck against the compliance checker and provide an assessment about criticity and policy compliance...and trigger an automatic hitl for admin if needed otherwise it allows approve the pr an apply afterwards

Makes sense, sounds like a solid setup. If your agents ever start calling cloud/internal APIs directly instead of going through the PR flow, that's exactly what we built Alter for, and we'd love to help. Either way, good luck with it!

what does the agent do while it’s waiting on the approval to happen?

It doesn't sit there blocked. The call comes back right away with a pending status and an approval id, so the agent can tell the user "this needs sign off from X" and get on with other stuff. Then you either await it inline with a timeout or stash the id and have a worker poll it later. Once it's approved we run the exact request that was approved and the agent gets the provider's response back. If nobody answers in time it expires (10 min by default, you can set it up to 24h) and the agent has to make the call again.

Reply on news.ycombinator.com