Yes exactly, so the only way to have a non public database is to run a Windows vm on AWS, I would rather avoid that, especially because we need one vm per customer.
Thanks for the detailed write up!
On that note, you don't have to use the official data gateway application. It certainly has a lot of ancillary benefits, but isn't critical. You can just as effectively create a bastion linux host and use ssh to remote forward the connection to AWS database[1]. I've done that plenty of times in the past, to access databases like Redshift and Dynamo that were within non-public VPCs. Easier to manage and cheaper to run per client, if that's a hard requirement.
You could also use something like QuickSight, if you're purely AWS. But I'd only recommend that if you expect to allocate a large amount of development resources for support. PowerBI is a lot more user friendly for business users to be self-sufficient, and sometimes it's less of a hassle to deal with the up-front complexity of that than to deal with the non-stop application support needs of business users.
[1] https://www.ssh.com/ssh/tunneling/example#sec-Remote-Forward...