I'm not sure where the idea that we took this down came from, but I checked with legal and we didn't. Such tools, services, etc. have existed forever. Just one reason why we encourage people to protect their public IP (https://developers.cloudflare.com/fundamentals/get-started/s...) and have Cloudflare Tunnel (https://developers.cloudflare.com/cloudflare-one/connections...).
There is no reason for them to scan the internet. They could simply probe the configured origin server from an IP outside the whitelisted cloudflare IP range, and display a warning if it's accessible.
RE “as long as no one leaked their IP” - the IPv4 space is quite small. It’s trivial to scan it and discuss unadvertised, but ultimately very public, servers.
If customers don’t already have an understanding of both of these points, then they need to increase their competence in areas that are, frankly, pretty basic.
Simple, they could scan the internet like I explained and notify their customers who’s site IP is findable this way with a big scary warning message. They could do this easily and cheaply, but for some reason they don’t.
Cloudflare are providing the service they say they are, it’s the customer’s fault if they don’t understand basic best practice.
My house has a lock on the front door. Yet that security can be completely undermined if a teen throws a brick at my window. That isn't the fault of the manufacturer of the lock on my front door.