We have done some interesting studies on infant memories. At first, you are correct that they basically can't remember anything.

Even then, though. At a very early age, they will recognize their mother. This comes well before "object permanence," which happens at about 8 months.

(Object Permanence is the ability to remember a thing even after you can't see it anymore, and to recognize it as the same, rather than a new one, when it returns. It is the ability to mentally follow an object when it "goes away" and to understand that it still exists)