Yes they can. If the token you give the LLM isn't permitted to access private repos you can lie all you want, it still can't access private repos.

Of course you shouldn't give an app/action/whatever a token with too lax permissions. Especially not a user facing one. That's not in any way unique to tools based on LLMs.