I know this probably seems like such a small detail to a lot of people, but I really love that the author adds comments.
I can't stand reading PyTorch or other neural network code and asking myself, "What architecture am I looking at here?" or "What the hell are these operations for?"
It's always like an mash up of reading some published paper code with deep effort behind it along with all the worst programming practices of complete unreadability.