Take a look at the Codex App Server or the Claude Agent SDK as a way to communicate with harnesses running headlessly and build your own UI on top. I’m sure Pi can be set up in a similar fashion.
I wanted more control over the harness itself, especially tool execution, sandboxing, approvals, and context management.
Recently built a native macOS app with GPUI on top of the same agent runloop. It's been interesting exploring how the same engine works across terminal and desktop workflows.
I have 2 versions that are just for working on unity projects. One that is for working on the client project. There is also one that I refer to as the ring of power / Death Star that I keep mostly to myself. Eventually, the client saw what I was doing and decided they wanted a custom ChatGPT.com experience for the whole team, so I got paid to build the most recent one.
Using AI to assist with the development of this stuff is the RSI you've been looking for. I'm in a situation where I was only able to solve a problem due to two levels of bootstrapping enabled by custom harnesses. The product the client uses cannot be "one shot" using the latest frontier tools. I had to build an ad-hoc agent loop fit for just that one purpose using another tool that was possible to build via copilot. For example, using copilot in VS over a unity project's scripts might as well be a drunk person throwing darts from a boat compared to a purpose built harness that can execute ad-hoc scripts in the editor environment as often as it pleases.
The client's custom experience is better than ChatGPT.com in many important ways now. It certainly performs better. It will run on safari without doing horrific things to layout and my CPU. You can easily exceed the capabilities of the frontier AI companies in ways that are important if you use their tools cleverly. These companies are great at making AI models. Not harnesses, websites, apps, etc.