HNHacker News
TopNewBestAskShowJobs

Andydy25

27 karma · joined December 3, 2021

Building Respan: an open-source Rust AI gateway built around plugins, with hosted observability and evals.

https://github.com/respanai/respan-gateway

submissionscomments
Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
Yeah, ours are more like GStreamer’s statically compiled plugins. We might consider WebAssembly in the future if there’s a need for it.
Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
We do have an official build on GitHub Releases with first-party plugins built in.
Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
We had a Docker image, but pulled it before launch because some APIs might still change. You have to build the Docker images, but we have GitHub releases for binaries.

We have configurable template plugins for common things like adding vendors and formats. No rebuild needed for config changes, but if you want custom logic beyond those templates, you need to compile it in.

And yes, crates still have supply-chain risks. I just meant you can review the source you’re adding, unlike a binary you don’t know.

Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
Yes, we used Claude to generate code, but we reviewed every line ourselves. For MCP, we considered rmcp, but our gateway needs hooks at each processing stage, so we would still have needed substantial custom logic. We ultimately implemented the MCP support ourselves.
Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
The plugins are natively compiled Rust crates, not interpreted or dynamically loaded.

We considered WASM and libloading, but plugins often require database connections or other privileges. With WASM, supporting that would be the SDK’s responsibility.

Dynamic loading with libloading didn’t offer enough benefit for this design, so we chose modular crates compiled into the application.

Andydy25··on Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
In Respan, plugins are not arbitrary executables that you download from a random repository. They are modular crates that you compile into your application.

Plugins offer better interface boundaries. For example, you can implement a PII feature without having to touch the logging, billing, or provider subsystems.