Not a physicist but I was explained it was like this:

Anything that has a mass close to the mass of the neutron would work better because we'd want to have a lot of elastic collisions to slow neutrons down. So something with lots of hydrogen would work, since the nucleus of hydrogen is a single proton so about the same mass as a neutron. It would mean water works. But so does polyethylene and it can be molded into various shapes and doesn't leak like water.

Another way to think about it is lead is heavy nucleus, imagine it like a bowling ball. A neutron on other hand are more like tennis balls. A fast flying tennis ball hitting a bunch of bowling balls will not slow down too much and will keep bouncing around at at high speed.

Now imagine this fast tennis ball hitting lots other tennis balls. Then the collisions are more elastic and it will slow down the neutron quicker.