Hacker News Reader

It's an interesting idea. But I guess there are would be two main problems.

1. Memory is limited. For this task you need to save a lot of words to make possible use it in applications. To solve this problem you should save words in a huge dimensional vectors.

2. This network can't solve the problem of missing letters in words with a high accuracy.

View on HN

I was thinking of encoding words as letter pairs. E.g. the word "hello" would have 1 'he', 1 'el', 1, 'll', and 1 'lo'. That seems more natural and robust than the binary bits of it's ASCII encoding.

There are two main advantage of ASCII approach. The first one is that vectors weight is not very big and you can build relatively small matrix that easy to store and faster to use. The second one is that usual encoding alternatives give a vary sparse vectors and as a consequence there would a big number of negative values (because we encode 0 as -1 value) inside the weight. For instance, if you have something encoded as 1 (word, letter or anything like that) with a big number of zeros (which would be encoded as -1 value in Discrete Hopfield network) that would be very unlikely to see the same ones at the same place multiple number of times and after some number of iterations negative values will reduce value encoded as 1 inside of memory and these -1 values would be as a dominant value for specific pattern.

Reply on news.ycombinator.com