analog chips could also be very interessting instead of using digital signals and processing them against the weights in the ROM. I have no idea if that scales with such big models though.
Nonetheless, yes, there are already implemented solutions for small NNs (I understand mostly acting as triggers).