I pay taxes to the US (and not to the country I live in) on the small amount of passive income generated in accounts located in the US. I'm not 100% sure if this is the Right Thing to do or not, but it seems fair. But my taxes would be a lot simpler and less stressful if I only had to report this income (and on years where this was 0, not report anything at all).

I had to pay. I have lived abroad for several decades and when I decided to do some independent contracting I realized that now I have to pay social security. Also, while I generally pay more taxes than an American, every once in a while there is a tax break that I get that Americans don't. In that case I have to pay that amount to the US.

You end up paying quite a bit to owe 0, because it's not easy to find a competent accountant who can, say, understand your Italian tax returns, and then file in the US based on those.