Seems like the trick is to use the LLM to produce a query - then the data you get back is simply the result of the query, and you can examine the query to check that it's appropriate.
I have an open issue about that here: https://github.com/simonw/datasette-chatgpt-plugin/issues/2
More about my explorations: https://simonwillison.net/2023/Mar/24/datasette-chatgpt-plug...