A Man Who Tried to Redeem the World with Logic | Hacker News Reader

A Man Who Tried to Redeem the World with Logic

nautil.us

122 pointsandreshb16 comments

I'm absolutely blown away by the fact that the possibility of recurrent neural networks encoding memory was discovered and published before electronic computers existed, and that the construction was entirely theoretical, based on first principles alone.

Logic is old. Transistors are recent, but you can compute with lots of other things. You can make a recurrent network with memory using basic graph theory and algebra.

The article seems to be describing something more like traditional computer memory (encoded in an electrically stable configuration of switches) than the memory of a recurrent neural network (encoded in the pattern of underlying connections). In biological terms that might be comparable to the distinction between short-term memory and long-term memory.

"Not only did Russell write back [about the proposed corrections to be made to Principia Mathematica], he was so impressed that he invited Pitts to study with him as a graduate student at Cambridge University in England. Pitts couldn’t oblige him, though — he was only 12 years old."

McCullough interview video: https://youtu.be/MTmR6X2w8Tg

He seems quite the figure.

Genius is often coupled with depression; sometimes in order to see the world is different light you need a different brain to see it with. I often wonder how a genius like Pitts can appear out of common people with no apparent genius in them - genetics is often a cruel process where amazing results are often coupled with not so amazing side effects.

> In a way Pitts was still 12 years old. He was still beaten, still a runaway, still hiding from the world in musty libraries. Only now his books took the shape of a bottle.

In an otherwise fine article, this was a staggeringly insensitive statement about the man's struggle with depression. That too at an age where it was even more poorly understood than today.

> The higher the probability, the higher the entropy and the lower the information content.

Is this a known and/or understood statement? It seems to be saying a lot for so few words. Would love any other sources that could expand on it.

It is known if we fix it to read "the lower the entropy".

Information Theory equates entropy with with the (expected) negative logarithm of probability.

I read that article at least a year ago before, but "past" doesn't show a previous mention here on HN. A bug?

>This article was originally published in our “Information” issue in February, 2015.

Ah, thanks.

Didn't this same thing happen between Wittgenstein and Russell two decades earlier?

Thank you for sharing this article.

How sad.

I don't know why I don't have nautil.us bookmarked.

Reply on news.ycombinator.com