Local models requires investment of about $2500 into hardware, so that you can run models at decent tok/sec. You then need to basically write custom harnesses. All of my stuff runs on a llama.cpp backend on a dedicated inference machine, and a client runs custom code built with OpenAPI library to make inference calls to accomplish things.