Not necessarily. Having the submit button first means it's the first action the user sees since most people (at least Westerners) read left-to-right. However, your reasoning with having the order the opposite with the submit button last is spot-on.

No specific advantage has actually been attributed to either choice of order. What matters is (1) keeping the order consistent throughout an application, and (2) following the provided style guide for the platform you're developing on.

(Also, the order may need to be changed if the primary action is destructive, such as a "Reset" button.)

I did an extensive write-up about this topic if you're curious on more details: https://dev.to/sergix/ux-illuminating-intention-198k