You'll continually get a raise each year and a portion of that will be new RSUs that vest over the next four years. If you ever want to leave, that continually means leaving a substantial amount of money on the table. Thankfully some companies will give you a hiring bonus equal to your unvested RSUs though (edit: the big tech companies poaching from each other will sometimes do it, for those asking).
As with all things labor related, RSUs are transactional. "Loyalty" shouldn't play into it because a large publicly traded corporation can only be so loyal to every one of it's employees.