You are absolutely right. I have a built a handful of complex interconnected LLM calls, and the thing I am dying for is a simple way to inspect what is happening at each step. For instance, one chain has 5+ steps where data is fetched, transformed, sent to an LLM, and transformed again. I have a homegrown solution to seeing what's going on, but it is missing a lot of what I would want to see.

I've seen some attempts at providing RAG tracing as a service, but for my specific use case, where in some instances I am making 100+ LLM calls per chain, it isn't a fit quite yet.