227 karma · joined February 20, 2018
Smarter model can figure out more sophisticated attack when following an injection . I believe in non-determinitic defence: each action or input to agent can escalate context sensivity. More sensitive context -> less risk your agent can take.
I find Bell-LaPadula model from 1970 (https://en.wikipedia.org/wiki/Bell%E2%80%93LaPadula_model) pretty interesting for that approach
Archestra is an open-source AI gateway and MCP orchestrator. It runs MCP servers in sandboxed environments, applies deterministic context-aware policy at the tool-call level, and handles the auth, audit, cost tracking, and observability, so an in-house AI team can let business users connect to real internal data. Now we are raising next round to expand.
We’re strong believers that there should be an independent, battle-tested, cloud-native, open-source platform that's free from big players and takes care of all that boring stuff. Something like GitLab, Grafana, or Clickhouse.
Repo is here: https://github.com/archestra-ai/archestra Matvey (CEO), Joey (Co-founder), and I are all in the thread. Ask us anything, including the unflattering stuff.
We figured out a workaround to limit activity to prior contributors only, and add a CI job that pushes a coauthored commit after passing captcha on our website. It cut the AI slop by 90%. Full write-up https://archestra.ai/blog/only-responsible-ai
The main LLM does have access to the tools or sensitive data, but doesn't have direct access to untrusted data (quarantine LLM is restricted at the controller level to respond only with integer digits, and only to legitimate questions from the main llm)
It’s really a cat-and-mouse game, where for each new model version, new jailbreaks and injections are found