Why not? The whole point of uploading the files is for retrieval/RAG. Consider viewing the files as nothing more than bypassing the RAG/Prompt step (or returning the entire RAG) and simply downloading them directly.

In what case would something contained in the files not be wanted to return to the user?

Read OpenAIs tos, nothing you build there is proprietary, it belongs to OpenAI. Do not use OpenAI products for proprietary work, you have no legal grounds to defend your IP, you should have read the fine print.