I think that is a security feature.
Otherwise, I can impersonate anyone with any name and contact image that I want. The way it is set up, you are the only one who has control over the displayed name (with fallback to the phone number). I actually like it that way.