This is the commitment and consistency effect.
The extreme cases can be found in cults where followers cling tighter to their beliefs once exposed.
A great example are the followers of Harold Camping, the Christian radio broadcaster who predicted the end of the world a few years ago, and kept re-predicting when it never came.