- create an "internal" domain for your lan, e.g. home.example.com
- make AWS Route53 handle that zone
- create some AWS IAM credentials for Route53
- create some hosts (e.g. router.home.example.com)
- use certbot with the route53 dns option to get a certificate and private key. Certbot will automagically add some TXT records to verify hostname ownership and provide you with that.
NO NEED to expose anything on your public IPs, but mind you: your hostname will appear in public CT logs. No "greatnascontainingmypartnersnudephotos.home.example.com" hostnames!