Sounds too me like you're saying "Let's take Hetzner so we don't have to be smart" and your CTO is telling you to be smart. In a way, he trusts you're smart enough to figure out a better way then just throwing hardware at the problem.

Also, while cloud pricing can be daunting, once you reach a point when Azure costs to much, you can look at optimizing things. There are a lot of techniques to keep cloud cost acceptable. Look up "FinOps" if you haven't already.