I also have this question, yes it should be. The forward pass should require accessing all the weights AFAIK.