"This seems really simple and obvious to me. So what am I missing?"

Personal identity is a deep philosophical problem - see Derek Parfit's 'Reasons and Persons'.