Possibly a regional thing, but among Massholes it is considered rude to "watch" people you don't know for a sustained period of time. It wouldn't be considered rude or "harassment" to call someone out for doing this.