I don't think most people think to do either direction by hand; it's all just matrix multiplication, you can multiply them in whatever order makes it easier.
For vector valued functions, the naive way you would learn in a vector calculus class corresponds to forward mode AD.