Production is not affected, the chatbot works with thin clones provided by Database Lab https://gitlab.com/postgres-ai/database-lab/.
In general, you are right. If, in your case, using the "raw" databases is unacceptable, it is better to anonymize data first (of course, the physical layout will change in this case).
However, Joe doesn't reveal the data. Engineers see only EXPLAIN (ANALYZE, BUFFERS) plans.
It may be considered acceptable. Of course, there are ways to retrieve particular values with some tricks like "explain analyze ... limit (select value from...)"), but this will be noticeable. Massive data leaks are not possible, Joe filters the output, presenting only metadata -- the whole idea was to provide the data for backend engineers, many of those don't have access to production data.
The question you're raising is broader: let's think how many times we send some personal data to our colleagues in Slack, how is it controlled now? are we okay with that? Emails, tokens, and so on.
At the same time, Joe does boost the development speed because it becomes easier to troubleshoot SQL performance and discuss it with colleagues, collecting reliable facts.
What would you say if it would be: 1. A chatbot for on-prem Mattermost? 2. A specialized GUI, SaaS or standalone?