Even at 500Da or less, it's much, much larger than that.
You may be interested in the work of Jean-Louis Reymond's group, who have done more or less exactly what you suggest: https://gdb.unibe.ch/downloads/
GDB-11, with 11 heavy atoms, has 26.4M structures (110.9M stereoisomers -- molecules aren't 2D). Going up to 13 gives you 970M molecules. Going up to 17 (still mostly below 250Da) is 166,400M.
There's a lot of space up there below 500Da.