I have written a small series of posts which show (with some simple Python code and math) the relationship between the training procedure of an ANN (backpropagation) and some simpler, more basic machine learning algorithms (like logistic regression):