Shameless plug: we're are building in the same space. We built an open-source platform for experimenting and evaluating LLM applications. Compared to LangSmith we are framework and model agnostic and are self-hosted. Here is our repo:
http://github.com/agenta-ai/agenta