That's why AWS also started offering Key Management Service: https://aws.amazon.com/kms/

You don't get your own HSM but it's MUCH cheaper ($1/key/month) and more scalable and available than an HSM.