I'm not sure a 'Cancel' button is the right UI, because settings pages are a confusing mess - sometimes a change takes place right away, other times you need to press the 'OK' button first. Perhaps an 'Undo' action is the better approach, just like making changes in a document?