I don't know if this is the real reason, but it's more natural for stack-relative addressing to have the stack grow downwards, because you can write e.g. [rsp+10] instead of [rsp-10].
Apparently on PA-RISC (hppa) the stack grows the other way, so it is arbitrary.