So it isn't a huge deal. It's the overall package that adds up to a much faster SOC.
It's a subtle distinction that can be hard to communicate, but while "64-bit" is not all that interesting, "moving to 64-bit" is in this case.
A lot of the improvement wouldn't be possible without the new architecture. In theory, you could do a new 32-bit architecture that has all the same benefits, but that would be terribly pointless.
Really, it's not just the SOC.
As described in @mikeash's piece, the 64-bit address length has allowed Apple to inline some per-object information (like reference counts) with object pointers, which is a big win in some key operations (2x faster object creation/destruction), which could sometimes make a big difference at the application level.