HNHacker News
TopNewBestAskShowJobs

misterman0

339 karma · joined August 4, 2018

I sound wierd I agree. And I am. You will see. But I'm mostly an ok dude. I try to be funny but sometimes I'm rude. It's all in the jest of becoming immune to HN sucking karma from your fucking fumes.
submissionscomments
misterman0··on Show HN: Didyougogo – An Altavista slayer
If you submit Google to the "gogo" index it should start to appear when you query.

Did you submit both a query and a URL?

Did you go go?

http://didyougogo.com/add

misterman0··on Show HN: Didyougogo – An Altavista slayer
Yes absolutely. I have been holding off crawling because I have no server capacity yet. That will probably sort itself out pretty soon from the looks of it. When I have the disk space I'll start using their data.
misterman0··on Show HN: Didyougogo – An Altavista slayer
As preparation for this demo, yes I absolutely should have run such a test. Eagerness won.

Thanks for the link.

misterman0··on Show HN: Didyougogo – An Altavista slayer
Thank you!

Yep, I have probably messed up the relevancy a bit because of constantly experimenting with how to load the model/index. Right now I'm using phrases (sentences) as well as words, both extracted during the tokenization process. Initially I used only phrases because using the current 65K vector-space model that would match any word to any phrase containing that word. There are perhaps sideeffects of reinforcing each word like that.

"long way to go"

I don't think so. The real bitch was to figure out how to maintain a good representation of the language model on disk. How to update it. Remove data from it. Now I anticipate a couple of months fine-tuning the balancing of the tree and testing relevance. From what I have heard so far, relevance is a little sub-par.

Scaling is the next thing. I have a great plan for that of course, mentioned somewhere in this thread.

misterman0··on Show HN: Didyougogo – An Altavista slayer
Ok. Hmm, I really thought my perfect was English.
misterman0··on Show HN: Didyougogo – An Altavista slayer
Yes, indeed. I used the following titles to test my boolean logic:

[0] First Blood

[1] Rambo: First Blood Part II

[2] Rambo III

Query:

title:first blood

-title:rambo

Result:

[0] First Blood

(the best one in the bunch)

misterman0··on Show HN: Didyougogo – An Altavista slayer
I thought it was a nice homage to DuckDuckGo. What it really means is "Did you submit both a query and a URL?"
misterman0··on Show HN: Didyougogo – An Altavista slayer
The catch is: this is 0.1 alpha software. I need a small team and some server capacity to get rolling. I need people to submit URLs. And a few hundred queries per second. That would scare the living shit out of big league search engines and might wake up some investor wanting to throw money at this.
misterman0··on Show HN: Didyougogo – An Altavista slayer
I was just talking to someone about scaling so I'm reusing what I said:

Scaling out technically and socially seems a little bit related. I want to scale out like this: a public search server (node) knows about other public nodes and the semantic topics their data carries. When a node cannot sufficiently answer a query it can reach out to other nodes by looking up a map of topic/list of nodes. Sharding by table/collection can also be solved the same way. That way, people owning public nodes can create queries that span tables they don't even host. They can build analytics using _their_ data _and_ the world's data. That's super-powerful.

misterman0··on Show HN: Didyougogo – An Altavista slayer
No this was not meant to be funny as in "haha".

You are right though, the index had only a handful of entries an hour ago.

misterman0··on Show HN: Didyougogo – An Altavista slayer
Thanks!

1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl.

2. I'll look into it. Thank you.

misterman0··on Show HN: Didyougogo – An Altavista slayer
My boolean logic is here: [1] I'm sure it has flaws.

Since the index had only five or six entries a couple of hours ago I set the matching to be wide instead of narrow. I'm also experimenting with loading the model with phrases, phrases and words or words only. I might have f-ed up the query parsing because of that. Remember, this is 0.1, fresh out of the press.

Searching the tree is here: [2]

Tokenization is here: [3]

[1] https://github.com/kreeben/resin/blob/master/src/Sir.Store/R...

[2] https://github.com/kreeben/resin/blob/master/src/Sir.Store/V...

[3] https://github.com/kreeben/resin/blob/master/src/Sir.Store/L...

← PreviousPage 7 of 7