I've designed my own as well but just went with the traditional key matrix with a diode per switch. Works well enough for the current size.
I made two videos talking about how I design my analog keyboard PCBs and how I'm using analog multiplexers here: https://youtu.be/TfKz_FbZWLQ
They do avoid having a diode per key, in exchange for needing a chip for N keys, where n is the number of of parallel inputs in your parallel to serial shift register.
They can also save IO lines. Obviously with if using chainable shift registers, you can have an unlimited number of keys with just 3 IO lines.
More realistic of course, would be to use several IO lines in parallel. For a 64 key matrix, one could say use 8, 8-bit shift registers, each connected to one input line, plus one output latch line and one output clock line (for a total of 10), vs the 16 lines needed traditional matrix.
8 input lines is nice, because if you pick the I/O lines right, retrieving the values is just a single port read. Making the procedure: latch, (toggle clock twice, read port, write to next location in memory buffer) x8.
There are of course downsides, like shift registers needing more board space than tiny surface mount diodes, but this is not necessarily a deal-killer.
Searching again, looks like zmk added support for the common 595 shift register: https://github.com/zmkfirmware/zmk/pull/1325
Edit: You could avoid using diodes, but diodes increase the number of switches possible via a matrix more than shift registers do by giving you more gpio. I’d imagine one can use both together.