You’re correct in thinking that there is nothing magical about LSB or MSB. As long as everything agrees, we get the results we’re expecting. Though for serial communication, networking, and similar, don’t assume anything :)
Historically, order is a little more interesting. Our earlier computers conveniently had instructions, registers, and memory all exactly the same width (perhaps 27, 36, or 60 bits). MSB/LSB was not a thing. But shortly after the “dawn of time” computers started to provide double- triple- and quad- data and instructions. Memory was expensive and if most data/instructions fitted into X bits (maybe 12, 32 or 36 bits) that’s what you used. But some data/instructions need more space, hence “double” floats, “long” ints, or complex instructions. Now designers (and programmers) needed to consider which part of the data/instruction is at which address, and order becomes a thing.
Each designer found advantages to a specific order … they tended to be small advantages and depended on their previous design decisions. But the tech was being pushed to the limit, so small advantages counted. For example, if you store instructions with the opcode at the “start”, then decoding can happen while the normal increment-PC mechanism is used to fetch the rest of the instruction – saves having extra logic to calculate the address of the rest of the instruction – saves a few gates here and a register there.
Computer history is marvellous – I’d recommend Tanenbaum’s “Structured Computer Organisation” and Siewiorek’s “Computer Structures”.