HNHacker News
TopNewBestAskShowJobs

michael_nielsen

8,027 karma · joined July 24, 2007

https://michaelnotebook.com
submissionscomments
michael_nielsen··on Improving the way neural networks learn
This is addressed in the marginal note attached to the sentence you quoted.

The essential point is that we're considering classification problems, for which the output is intended to be 0 or 1. I address the more general case of regression problems (where y may take any value) in a later exercise.

Hope that helps!

michael_nielsen··on Improving the way neural networks learn
The OP is quoting LeCun, so this is not a coincidence!
michael_nielsen··on Improving the way neural networks learn
Which statement about cross entropy is confusing?
michael_nielsen··on How to Make a Complete Map of Every Thought You Think (2003)
Thanks for the pointer, this is great. Tip for people without a subscription to Safari: many public libraries have subscriptions, and will allow online access. I'm reading mine through the Toronto Public Library site.
michael_nielsen··on How to Make a Complete Map of Every Thought You Think (2003)
I've read it before. To me, it looks unedited, and the product of an original mind, but not a kook. A good editor could quickly turn the text into a remarkable essay.
michael_nielsen··on Inside Google Brain
Could submissions from Wired (& other sites which practice link-baity titles) be automatically put in a moderation queue? They wouldn't appear in the "New" queue until the title had been reviewed by a moderator, and fixed if appropriate. That sort of fix seems to be necessary a large fraction of the time. Regularly having such titles on the front page is a small but real hit to HN's quality.
michael_nielsen··on What Problems to Solve
It's interesting to look at what may be a homepage for the recipient of the letter:

http://johnen.shinshu-u.ac.jp/~mano/profile_e.html

The letter is from an excellent collection of Feynman letters, "Perfectly Reasonable Deviations from the Beaten Track". There's a thoughtful review of the entire collection (by Freeman Dyson) here:

http://www.nybooks.com/articles/archives/2005/oct/20/wise-ma...

michael_nielsen··on What Is a Bitcoin, Really?
Possibly you're thinking of my article "How the Bitcoin Protocol Actually Works": http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-a... It was on the HN front page 7 months ago.
michael_nielsen··on The power of lonely: What we do better without other people around
This is one of the most beautiful and insightful quotes about creative activity I've ever seen. Thanks for reminding me of it.
michael_nielsen··on Neural Networks and Deep Learning
Thanks for the clarification. Yes, it would be great to have that policy in the FAQ.
michael_nielsen··on Neural Networks and Deep Learning
Chapter 3 (of six) will be released mid-July. Apart from that, I haven't set a hard finish date. Some authors are helped by such deadlines, but my experience writing my previous two books is that (for me) such deadlines hinder more than they help.
michael_nielsen··on Neural Networks and Deep Learning
I ran an IndieGoGo, which is long since over. Other than the IndieGoGo I haven't given much thought to setting things up so people can contribute, that's very kind of you to offer. I'll investigate adding PayPal or some alternative when I have a little time. In the meantime, you could send me Bitcoin at: 17ukkKt1bNLAqdJ1QQv8v9Askr6vy3MzTZ It'd be much appreciated!
michael_nielsen··on Neural Networks and Deep Learning
I'm the author. Thanks for the support! And I'm glad you're enjoying the book.

The HN search results suggest that this is the second time the book has made it to the HN front page.

However, you may have picked up a different impression because the second chapter -- a largely self-contained introduction to backpropagation -- also made it once onto the front page. Given that I'm rolling out chapters as I finish them, this seems to me like multiple posts from the same blog appearing on the front page. Certainly, the individual chapters are (I hope) far more substantive than most blog posts.

(For the record, I had no idea this would be posted.)

michael_nielsen··on Performance of quantum computer no better than ordinary PC, say analysts
That's right. The search result, however, assumes the only way of accessing the database is through a black box oracle. That's why I wrote "non-oracle" in my previous comment. It's called an oracle because the internals of the black box are assumed to be unknown. That assumption is necessary in order to prove that classical computers need ~ N operations. If you know something about the internals of the oracle, then it may well be possible to exploit what you know to search faster.

The quantum search algorithm is very interesting. However, most people in the field would, I think, agree that a non-oracle separation between quantum and classical would be vastly more interesting. Oracle results feel a little too close to cheating.

There is, incidentally, a nice paper showing that for computing total Boolean functions the quantum oracle model can be no more than polynomially faster than a classical model. This means there can be no exponential speedup for such functions, which is a real pity.

http://arxiv.org/abs/quant-ph/9802049

michael_nielsen··on Performance of quantum computer no better than ordinary PC, say analysts
Shor's algorithm is exponentially faster than the best existing classical algorithm for factoring. But that doesn't mean there's not an exponentially faster classical algorithm which would be comparable in speed to Shor's algorithm. Some noted experts believe this is possible, and even likely. Richard Lipton believes it, and claims that Peter Sarnak does too:

http://rjlipton.wordpress.com/2013/04/27/sex-lies-and-quantu...

Yes, 't Hooft and others, including Leonid Levin and Oded Goldreich, believe that quantum computers won't function because of quantum mechanics breaking down. See, e.g.:

http://www.wisdom.weizmann.ac.il/~oded/on-qc.html

http://www.cs.bu.edu/fac/lnd/expo/qc.htm

michael_nielsen··on Performance of quantum computer no better than ordinary PC, say analysts
Your first paragraph is quite right. But the second paragraph is wrong: for (non-oracle) problems quantum computers are merely suspected to be faster in principle than conventional classical computers. It's never been proven.
michael_nielsen··on Beach theft
I was living in Brisbane, Queensland, at the time (in Ferny Hills, if you know it), and the house was a Queenslander. I imagine the house may well have been laundered through such a service.
michael_nielsen··on Beach theft
Growing up, I lived on an isolated 4 acre block, surrounded by forest. The only house nearby was an old house, built on stumps, a hundred meters or so away, with no-one living in it. One day a truck showed up, a team of workers put the house on the back of the truck, and took it away.

A year later detectives arrived at our house. They asked to speak to my mother, and told her that the house next door had actually been stolen, and the real owners were now looking for leads, hoping to find their stolen house.

michael_nielsen··on Bret Victor: Seeing Spaces [video]
It is not going to waste. Deep ideas are more durable than either code or startups, but they often take longer to have an impact.

As an example, "Inventing on Principle" has inspired many people and companies to think more deeply about development environments, and to try implementing more speculative ideas. A skeptic can point out that much of that consequent work is of mixed quality, but that's just Sturgeon's Law. What matters is that more people will think better thoughts about more important problems. And that's certainly occurring as a result of Victor's talks and essays.

michael_nielsen··on Framer: Prototyping Toolkit
On the "Questions" page it says that Chrome support is patchy, and Firefox / IE aren't supported at all. So far as I can see this isn't stated upfront on the front page, but is buried. Annoying.
michael_nielsen··on Framer: Prototyping Toolkit
Many of the examples seem partially or entirely broken under Chrome 34 (running on Windows 7).
michael_nielsen··on Google Maps Has Forsaken Us
I switched to Bing Maps a month or so ago. It's not as good as the old Google Maps -- there's less detailed information -- but at least the basic functionality works. Can't say that for the new Google Maps.
michael_nielsen··on Why CPUs aren't getting any faster (2010)
It means signals can only propagate about 6 centimeters during a single CPU clock cycle. That makes accessing information off-chip difficult in a single cycle, since 6 centimeters is on the same order as the linear scale of most motherboards.

This limit doesn't immediately impact computations that are carried out entirely on-chip. I expect that's the great majority of computations on today's CPUs, which typically have large caches. So I'm not sure the GP's explanation is correct, although it's certainly interesting.

michael_nielsen··on How I wrote Game Programming Patterns
Thanks!
michael_nielsen··on How I wrote Game Programming Patterns
Awesome post. I'm finishing my third book, and totally recognize the agonizing over whether the book will ever get done!

On not using a conventional publisher: my first two books were with mainstream publishers (Cambridge University Press and Princeton University Press). For my third book I decided to self-publish online only, because that makes it possible to integrate animations, interactive Javascript, and so on. Four publishers have asked about publishing paper editions of the book, but it's hard to imagine going back to the printed page as the primary target medium. Having the web as the primary target is so much more powerful.

michael_nielsen··on How the backpropagation algorithm works
Belated thanks, that's extremely helpful. I'll take a look at those links which are unfamiliar to me (I've already read the Nature of Code link, it's a great book.) Tidbits:

+ Yes, the code presented "works" for multiple hidden layers. But it converges too slowly to be useful, except for very small networks. Later chapters introduce new ideas which help backprop converge faster, and which start to make multiple hidden layers quite feasible.

+ On the broader machine learning front, I think scikit learn is a pretty nice general library: it's approachable, easy to use, and has reasonable docs. It'll be interesting to see how it develops.

+ For neural nets in particular, elsewhere in this thread Dave Sullivan has done a nice job distilling out a list of good libraries. I think it'll be a while before there's a real out-of-the-box solution for neural nets, though, since setting parameters for neural nets is something of an art.

michael_nielsen··on How the backpropagation algorithm works
Author here. I'd love if you could give me pointers to your favourite explanations of how neural networks function (even better, if you can say what you particularly liked about them).

(I really enjoyed reading the remainder of your comment.)

michael_nielsen··on Two Big Steps Toward the Quantum Computer
Faster-than-light communication using entanglement is not possible, according to the standard rules of quantum mechanics.

(Dropping some credentials doesn't seem inappropriate here: I worked professionally as a researcher on quantum computing and quantum information for 13 years.)

michael_nielsen··on How I Came to Write D
This has been posted many times, but the words of Teddy Roosevelt bear frequent rereading:

"It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat."

The critics often seem to be saying, implicitly, "_I_ could do it better". It's a claim of superiority. I guess it's beguiling, because criticism of this form is so much easier than genuine creation. I think many of the critics don't consciously realize this, since they've never created something of great worth, and have no idea how difficult it is.

I've known very accomplished people who could be haters. But usually accomplished people seem to be appreciative of creative projects, and understanding of flaws (while wanting them eliminated). I think it's because they understand just how difficult it is to do good creative work. They know that someone can bring immense gifts and effort to bear on a project, and still have it go awry.

michael_nielsen··on Emails From Schmidt And Sergey Brin Show Agreements Not To Hire Apple Workers
"If all the big companies just move each other's employees around, nobody benefits."

The companies may not benefit, but the developers certainly will! Not just the ones being moved around, either --- it'll raise market prices, and so _every_ developer will benefit, even those who don't move.

← PreviousPage 4 of 17Next →