Somewhat related, if someone want to build something awesome, I haven't seen anything that merges lucene with BPE/SentencePiece.

SentencePiece has to make it so you can shrink the memory requirements of your indexes for search and typeahead stuff.