Disclaimer: I work at AWS, but not involved in billing. The below is my personal experience having dealt with AWS outside of my current role there.
I agree its not worth it if you're just hosting a static page, but if you're using it for other development it can be worthwhile. (I have to admit that Cloudflare/R2 is looking promising for static hosting, although workers seem a bit more limited than lambda. Definitely an interesting ecosystem to keep an eye on)
Even thought it has unbounded billing, in the case of mistakes or unexpected surges, they're often willing to assist and even nix charges. Ultimately, they'd rather you continue using the service and pay future bills than have you quit while they try to collect on a bill you can't pay.
On my personal account, I have billing alerts configured low, and its billed via SEPA from a separate bank account that I use just for cloud providers (N26 makes it easy to configure a separate account with a distinct IBAN). In the unlikely event goes over the amount I have in that account, it'll fail to charge me and I can look into disputing the charges.