From Wikipedia:
When people began to multiply on the face of the ground, and daughters were born to them, the sons of God saw that they were fair; and they took wives for themselves of all that they chose. Then the Lord said, "My spirit shall not abide in mortals forever, for they are flesh; their days shall be one hundred twenty years." The Nephilim were on the earth in those days—and also afterward—when the sons of God went in to the daughters of humans, who bore children to them. These were the heroes that were of old, warriors of renown.
— Genesis 6:1–4, New Revised Standard Version
Basically says that there were people like humans, but different and larger and stronger. And that at some point they died out.Sounds a lot like what we know about Neanderthals.
Also Genesis is sort of the "deep memory" of humanity. Especially the first few chapters.