Somewhere I ran across the idea of setting a prehistoric fantasy novel in Doggerland, which is a big swathe of Europe that got submerged.

https://en.wikipedia.org/wiki/Doggerland

Neanderthals, Sapiens, and other close relatives like Denisovans could serve as analogues of fantasy races, like elves, dwarves, and hobbits. Non Neanderthals and Sapiens would have to be on the rare side to explain how they haven't been discovered today, but the fact that the entire region was submerged helps with that.

Perhaps certain factors could have made the region extremely prosperous by standards of the time and a nexus of prehistoric trade, giving rise to a rich culture attaining legendary status. However, none of its architecture used stone, and all of its knowledge was carried through oral tradition.