That seems very extreme. I remember my entire childhood from about 4 years old on.
Myself, I remember maybe one or two things per year from 4+ onwards if I focus; flashes of images. This gradually ramps up after ~12yo, and I can actually say I remember some events from around 16+ well enough to describe them and place in rich context. Properly detailed memory? That starts for me somewhere after 20. I'm 36.