I cannot find any such numbers in the paper. What the paper says is that MeZO converges much slower than SGD, and each step needs two forward passes.
"As a limitation, MeZO takes many steps in order to achieve strong performance."
"As a limitation, MeZO takes many steps in order to achieve strong performance."
No comments yet.