Did you submit both a query and a URL?
Did you go go?
339 karma · joined August 4, 2018
Did you submit both a query and a URL?
Did you go go?
Thanks for the link.
Yep, I have probably messed up the relevancy a bit because of constantly experimenting with how to load the model/index. Right now I'm using phrases (sentences) as well as words, both extracted during the tokenization process. Initially I used only phrases because using the current 65K vector-space model that would match any word to any phrase containing that word. There are perhaps sideeffects of reinforcing each word like that.
"long way to go"
I don't think so. The real bitch was to figure out how to maintain a good representation of the language model on disk. How to update it. Remove data from it. Now I anticipate a couple of months fine-tuning the balancing of the tree and testing relevance. From what I have heard so far, relevance is a little sub-par.
Scaling is the next thing. I have a great plan for that of course, mentioned somewhere in this thread.
[0] First Blood
[1] Rambo: First Blood Part II
[2] Rambo III
Query:
title:first blood
-title:rambo
Result:
[0] First Blood
(the best one in the bunch)
Scaling out technically and socially seems a little bit related. I want to scale out like this: a public search server (node) knows about other public nodes and the semantic topics their data carries. When a node cannot sufficiently answer a query it can reach out to other nodes by looking up a map of topic/list of nodes. Sharding by table/collection can also be solved the same way. That way, people owning public nodes can create queries that span tables they don't even host. They can build analytics using _their_ data _and_ the world's data. That's super-powerful.
You are right though, the index had only a handful of entries an hour ago.
1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl.
2. I'll look into it. Thank you.
Since the index had only five or six entries a couple of hours ago I set the matching to be wide instead of narrow. I'm also experimenting with loading the model with phrases, phrases and words or words only. I might have f-ed up the query parsing because of that. Remember, this is 0.1, fresh out of the press.
Searching the tree is here: [2]
Tokenization is here: [3]
[1] https://github.com/kreeben/resin/blob/master/src/Sir.Store/R...
[2] https://github.com/kreeben/resin/blob/master/src/Sir.Store/V...
[3] https://github.com/kreeben/resin/blob/master/src/Sir.Store/L...