Backpropagation step by step: the forward and backward pass

I went back over backprop recently and it was humbling how much this stuff had faded, especially nowadays with large models that encourage passive learning in my opinion. So writing it up in public was my way of relearning it.

I tried to make it intuitive, interactive and with some details and proofs for the curious, and a few exercises at the end to practise actively.

Feedback welcome, especially if a step is unclear or something is wrong.