100% agreed. This is very much human nature and impossible to avoid. I do this; you do this. We meet someone and very rapidly bucket them into whatever categories we've built over the years. When you're younger, it takes longer because those buckets aren't as well defined. But once I've played someone, it's very hard for them to move into another.
My favorite clip that encapsulates this thread is from Suits[0]. Mike hates it, but first impressions matter.