Babies have memories, some going back pre-birth. They can recognise music and voices from the womb. My daughter started understanding and following instructions around 3-4 months.
What you might be talking about are adult accessable memories. At some point, the structure of memory changes. I personally suspect it’s tied to the acquisition of language, but that’s just my opinion. The key point is that memories from before the transition are difficult, or impossible to interpret. This sets the line for “first memories”. For most it’s around 2-3 years, though it can be earlier (mine is my 1st birthday).
Interestingly, the better someone’s adult memory, the later their first memory tends to be. This implies multiple revisions, each one leaving older memories inaccessible, but giving better performance/sorting.