Not using Elastic Beanstack myself but you can definitely use Parameter Store across multiple dynamic instances, including ECS and Lambda.
Paths are now supported as well, so you could do something like prod/mykey and staging/mykey and set up IAM roles to limit to given environments.