I believe this is a misunderstanding. You can totally connect to AWS databases with PowerBI, assuming your AWS database is routable. You can't set up the connection through the web interface, but you can use PowerBI Desktop to set up virtually any database that has an ODBC driver as a source[1]. And once you publish it to the Azure-hosted PowerBI instance, it'll use that connection string to connect directly to the data source and automatically update/refresh like any other data source.
As for a VM running Windows Server - that's the option if your AWS database isn't routable by the Azure-hosted PowerBI service. You install a gateway for PowerBI to hop through, and configure all of the data sources within that gateway. That way your internal AWS databases are not publicly routable, and you can lock down and centralize the connection between PowerBI and AWS via that gateway box[2]. Data Gateway is a Windows-only application at the moment, so that part is a pain. But it's a relatively minor pain in the grand scheme of BI implementations.
[1] https://docs.microsoft.com/en-us/power-bi/desktop-connect-to...
[2] https://docs.microsoft.com/en-us/power-bi/service-gateway-ge...