When I first posted my reply every time I watched that demo the neural net line was converging on a line perpendicular to the line in the background.

(rereading my original comment I think I described what I was seeing incorrectly but this new reply correctly explains what I was originally seeing)