100% of the driving advice I've ever gotten is to always look over your shoulder before moving over, no matter what a glance in the mirror says. In all fairness, had I not been too close to the car in front of me, that glance would not have led to an accident.
I do have my mirrors adjusted as you suggest. From my experience with rental cars, I suspect that you and I are the only people in the world who do it that way.