If you’re in AWS, wouldn’t SSM agents on instances be preferable to SSH access? That provides for both access control (IAM for user access, SSM documents for constraining command execution authority) and auditing/logging of executed commands (CloudTrail).
This does not work for interactive terminal use cases, but does work (in my experience) if you’re targeting immutable instances. It also has the lovely side effect that you can create scheduled tasks within the AWS control plane (if that’s your cup of tea).
Example SSM client: https://github.com/itsdalmo/ssm-sh
Disclaimer: I’m implementing this in a large enterprise environment.