I'm not sure what the point of this is. Aren't there enough LLM playgrounds out there?
Maybe if they wanted to develop it into a useful feature it would create unique LLM benchmarks from your pull requests and issues, so you can easily test which model performs best on your codebase. Then augment that with fine-tuning to your code style, etc.