I do that all the time when people look at me. Look away like you are staring at something important; they very quickly try to see what you are interested in.