If you have one wallet with 17 addresses, doesn't this defeat the whole "your entire bitcoin cache can be protected by memorizing one 12 word phrase" aspect.
Another possible way would be to use a rotatable key derivation algorithm, where you can provide the 12 word phrase plus a number n, and it will provide you a different wallet for each value of n. Of course the key derivation algorithm must be quantam secure for the security benefit to apply in this context.