https://tile.googleapis.com/v1/3dtiles/root.json?key=AIzaSyB...
May want to confirm the map renderer is making its requests using the URL with both the session and API key instead of only the API key. If your API requests are base key only, every refresh client-side might be unexpectedly consuming one of your root quota units, rather than each user getting a session of unlimited tile loads for 3 hours. If you're seeing network traffic in the console from your renderer out to the API without the session token in the query string along with the API token, or if you see the session token is totally different after refreshing, that might be a sign of something wonky causing quota depletion.
Example URL for a tile fetch with both params:
https://tile.googleapis.com/v1/3dtiles/datasets/CgA/files/UlRPVEYuYnVs.json?session=SESSION_TOKEN&key=YOUR_API_KEY
https://developers.google.com/maps/documentation/tile/create-renderer#requests-and-responses