Not all of them. Some illusions are structured like relatively "normal" images, so unless you explicitly draw attention to the issue, people will just accept what they see and move on. See e.g. tower on a chessboard, or blue/pink dress thing.
The dress is sort of a variation on the same theme, except it divides people by how they extrapolate the rest of the scene.