Yes, any attacker can search for your site's DNS history and try to find the real ip there. For example you can get the history for some sites using this service: https://www.netcraft.com/

The good thing is that there's a real fix for this and is to configure your server's firewall to only accept requests from Cloudfare's ips.