Also, if the library removes stop words, does that mean its limited in which languages it supports? I'm assuming only English stop words are filtered
Not being critical here, just genuinely curious and don't have time to look through the code ATM
Also, if the library removes stop words, does that mean its limited in which languages it supports? I'm assuming only English stop words are filtered
Not being critical here, just genuinely curious and don't have time to look through the code ATM
Switching to a trie-like structure that compresses prefixes and suffixes can lead to significant savings. Building the structure can be a bit more burdensome, so there is a trade off there. There is a paper describing the approach [2] and, if you're interested, my JavaScript implementation [3][4].
[2] http://www.aclweb.org/anthology/J00-1002.pdf
[3] https://github.com/olivernn/lunr.js/blob/master/lib/token_se...
[4] https://github.com/olivernn/lunr.js/blob/master/lib/token_se...
The stop words that are being removed can be configured via `Wade.config.stopWords`, which is an array of stop words that Wade will remove.
Good thinking on making them configurable!