Privileged doesn't mean easy. Even if you're poor and white, you will never be judged by a potential landlord, store security employee, or taxi driver on the basis of your skin. You may have a tough life for all sorts of other reasons, but that won't be one of your problems, and that is a form of privilege.