Hacker News Reader

Dang only forward passes. The real secret was in the backward pass! I was also curious to learn how they implemented the dualpipe scheduler

Do they even have an optimized backward? It looks like optimizations like this aren't needed during training. Their V2 paper also suggests so.