HNHacker News
TopNewBestAskShowJobs

michael_nielsen

8,027 karma · joined July 24, 2007

https://michaelnotebook.com
submissionscomments
michael_nielsen··on Leslie Lamport awarded Turing Award
I like that the full news release (http://www.acm.org/press-room/news-releases/2014/turing-awar... ) comes as both HTML and a PDF built with --- wait for it! --- LaTeX.
michael_nielsen··on Billionaires With Big Ideas Are Privatizing American Science
Bill Press (the quoted adviser) is an outstanding physicist -- I believe he was the youngest physicist ever tenured at Harvard. I think the quote was out of context. He wasn't saying he doesn't like physics, he was saying it can be hard to sell as a political priority.

http://en.wikipedia.org/wiki/William_H._Press

michael_nielsen··on Show HN: I've been making one HTML5 game per week. Here's my 10th game
I really enjoy your games and (especially) the tutorial side. Great stuff! But I have been a bit put off by the frequent plugs for your newsletter. Of course, you couldn't have known that before commenting here, and so I hesitated a while before posting this. But I figure it may be a useful data point.

In any case, I hope you keep up the great work, I'll certainly be following!

michael_nielsen··on Show HN: I've been making one HTML5 game per week. Here's my 10th game
Here's an article with lots of interesting background info from the same author (lessmilk), "What I learned while doing my 'one game per week' challenge":

http://gamasutra.com/blogs/ThomasPalef/20140225/211663/What_...

Worth it especially for the following superb link:

http://www.gamasutra.com/view/feature/130848/how_to_prototyp...

michael_nielsen··on “That's a stupid idea”
Bessemer Venture Partners' anti-portfolio makes it clear just how hard it is to appreciate ideas early on:

http://www.bvp.com/portfolio/antiportfolio

They had the chance to invest in the following companies early, but didn't: Intel, Google, Apple, FedEx, PayPal, and others.

michael_nielsen··on Ask HN: Making extra income with books
I use a pretty vanilla LaTeX setup - edit in emacs, display using Yap, and version control with Git and (sometimes) Github. I use the MikTeX distribution for Windows, but use few advanced features - for me, LaTeX is pretty good out of the box.
michael_nielsen··on Ask HN: Making extra income with books
For tex -> html I use Mathjax to do the hard work, and a hand-rolled script to do the rest. I'm starting to experiment with d3 and AngularJS for features beyond what LaTeX alone can offer.
michael_nielsen··on Ask HN: Making extra income with books
I'm currently working on my third book, a book about neural networks and deep learning. Here's the first chapter:

http://neuralnetworksanddeeplearning.com/chap1.html

It'll be free online once complete, but I raised money with an Indiegogo campaign (about 10k). I'm self-publishing, in part because I wanted full control over the book. This gives me the ability to experiment with in-text videos, and with other tricks. For instance, when a reader clicks on an equation reference in the text, the relevant equation appears in the margin, as a reminder. Clicking on the marginal equation will take you to the context in which the equation originally appeared. This cuts down on tedious back-and-forth.

My two earlier books were both published in the traditional way:

+ A book for general audiences about networked science, "Reinventing Discovery: the New Era of Networked Science": http://www.amazon.com/dp/0691160198 Published by Princeton University Press.

+ A textbook about quantum computing, jointly with Ike Chuang: http://www.amazon.com/Quantum-Computation-Information-Annive... Published by Cambridge University Press. No e-book at all when first published (2000)!

Both were written with LaTeX, and included many illustrations. "Reinventing Discovery" was actually rekeyed entirely by the publisher, but the textbook was produced from our LaTeX copy.

michael_nielsen··on News is bad for you
I said Dobelli should be embarassed. He certainly should be embarassed by several of those examples.

But Taleb's page is also embarassing to Taleb. He's presenting many of these as his original ideas when, well, they're simply not.

michael_nielsen··on News is bad for you
I stopped reading at Taleb's first example. I've seen very similar stories from both Richard Feynman and John Littlewood. Here's Taleb:

"I was thinking about calling my third cousin Antiochus this morning when the phone rang. Miracle! It was him on the other line; this confirms my developed sixth sense! This is a great omen except that perhaps I should wake up and take into account the number of times when I thought about calling him without his calling me; the times when he called me without my thinking about calling him; and, most significantly, the numerous occurrences of my not thinking about him, and him not trying to call me."

Feynman has several variations on this theme. Here's one:

"I remembered the time I was in my fraternity house at MIT when the idea came into my head completely out of the blue that my grandmother was dead. Right after that there was a telephone call, just like that. It was for Pete Bernays-- my grandmother wasn't dead. So I remembered that, in case somebody told me a story that ended the other way. I figured that such things can sometimes happen by luck--after all, my grandmother was very old--although people might think they happened by some sort of supernatural phenomenon."

And another:

"You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing! "

There's an extended quote in the Feynman biography "Genius" (by James Gleick) which is a closer match than both these. I don't have an easy cut-and-paste, alas, but here's a link on Google books (it's the third page link):

http://books.google.ca/books?id=j42RD66g72oC&printsec=frontc...

Taleb basically reads as a reorganized and edited version of this last quote.

Edit: I couldn't resist reading on. Taleb's article is embarassingly bad. Half his "original examples" I'd heard before from statisticians and other scientists, often years before Taleb's books. I will say this: there are a few instances in there where Dobelli should be genuinely embarassed. But Taleb should be at least equally embarassed by presenting, Wolfram-like, well-known truths as his deep original thought.

michael_nielsen··on How the Bitcoin protocol actually works
> You've described a 51% attack.

No, this isn't related to 51% attacks.

Your other point I agree with; this is also discussed in the post comments.

michael_nielsen··on How the Bitcoin protocol actually works
Original post author here. Thanks -- your comment pretty much nails the answer to my question.

Edit: A problem with this is that it would be necessary for a naysayer to exhibit the other transaction (proof of double spending). But they couldn't forge such a transaction, since they don't have the private key necessary to generate the signature. So I'm still a bit puzzled by this.

Edit 2: And it appears that bcoates is making the same point.

michael_nielsen··on Thoughts on Bitcoin
You can remove the log scale:

https://blockchain.info/charts/estimated-transaction-volume?...

I used the log scale to make the point that transaction volume is relatively slow-moving on a log scale. That is certainly not the case for the exchange rate!:

https://blockchain.info/charts/market-price?showDataPoints=f...

Incidentally, the volume graph has a notable spike in December 2011. I've Googled and asked around a bit, but don't have a good understanding of what caused the spike.

michael_nielsen··on Thoughts on Bitcoin
The following graph shows the estimated volume of transactions:

https://blockchain.info/charts/estimated-transaction-volume?...

You see immediately from the graph that the growth in number of transactions is relatively slow, considering all the media attention, and certainly far, far slower than the increase in the exchange rate.

Note that this graph is not the same as the number of "legitimate transactions" mentioned in the OP. But it's at least consistent with the point of view suggested in the post.

michael_nielsen··on A universal income is not such a silly idea
See http://en.wikipedia.org/wiki/Guaranteed_minimum_income#Minim... for some pointers to schemes which have actually been tried.
michael_nielsen··on How I saved $39,420 by not buying a domain without a prototype
Why, just this morning I saved myself 360 million dollars by not buying a Boeing-747!

http://www.boeing.com/boeing/commercial/prices/

michael_nielsen··on Using neural nets to recognize handwritten digits
Thanks for the tip, I'll look into it!
michael_nielsen··on Using neural nets to recognize handwritten digits
Networks of perceptrons are universal in the standard sense used when talking about circuits --- they can compute any finite Boolean function.

I agree that the relationship between circuit complexity and Turing machines is somewhat subtle, for the reasons you mention. The relationship is greatly clarified by the notion of uniform circuit complexity, which makes it possible to prove an equivalence between a (carefully defined notion of) circuit complexity and Turing machine complexity. Unfortunately, I don't know of a good online treatment of uniform circuit complexity. I learnt it through a 1993 paper by Andy Yao, but that's definitely not a good introductory reference!

In any case, in my book I'm using the term universal in the same way as people usually use it for circuits, i.e., it means the same thing as when people say that the NAND gate is universal for computation. Hope that clarifies things.

michael_nielsen··on Using neural nets to recognize handwritten digits
My current plan is to describe some pretty recent results -- most likely, the big breakthrough on ImageNet by Krizhevsky, Sutskever and Hinton (http://www.cs.utoronto.ca/~ilya/pubs/2012/imgnet.pdf), which uses convolutional nets. I may also describe the famous Google-Stanford "cat neuron" paper (http://ai.stanford.edu/~ang/papers/icml12-HighLevelFeaturesU... ). But at this point things are moving so quickly that I'll keep my options open, and if more exciting things come up, I may change my plans.

Of course, there's a tremendous amount going on, so my broader philosophy is to focus on fundamentals. Readers who thoroughly master the core ideas shouldn't have much trouble later getting up to speed with the result-of-the-month.

michael_nielsen··on Using neural nets to recognize handwritten digits
They're discussed later in the book. The first chapter is an introduction, and I didn't want to introduce convolutional nets before (for example) fundamental techniques such as stochastic gradient descent and backpropagation.
michael_nielsen··on Could Media Get Too Smart about Consumers?
That's blogspam from Forbes. Actual link: http://www.nytimes.com/2012/02/19/magazine/shopping-habits.h...

Based on Charles Duhigg's very good book about habits.

michael_nielsen··on Majority is not Enough: Bitcoin Mining is Vulnerable
The key idea is very simple and is very clearly explained on pages 6-7:

"When the public branch is longer than the private branch, the sel fish mining pool is behind the public branch. Because of the power di fferential between the selfi sh miners and the others, the chances of the sel fish miners mining on their own private branch and overtaking the main branch are small. Consequently, the selfi sh miner pool simply adopts the main branch whenever its private branch falls behind. As others find new blocks and publish them, the pool updates and mines at the current public head."

"When the sel fish miner pool finds a block, it is in an advantageous position with a single block lead on the public branch on which the honest miners operate. Instead of naively publishing this private block and notifying the rest of the miners of the newly discovered block, sel fish miners keep this block private to the pool. There are two outcomes possible at this point: either the honest miners discover a new block on the public branch, nullifying the pool's lead, or else the pool mines a second block and extends its lead on the honest miners."

"In the first scenario where the honest nodes succeed in finding a block on the public branch, nullifying the sel fish pool's lead, the pool immediately publishes its private branch (of length 1). This yields a toss-up where either branch may win. The selfi sh miners unanimously adopt and extend the previously private branch, while the honest miners will choose to mine on either branch, depending on the propagation of the noti fications. If the selfi sh pool manages to mine a subsequent block ahead of the honest miners that did not adopt the pool's recently revealed block, it publishes immediately to enjoy the revenue of both the first and the second blocks of its branch. If the honest miners mine a block after the pool's revealed block, the pool enjoys the revenue of its block, while the others get the revenue from their block. Finally, if the honest miners mine a block after their own block, they enjoy the revenue of their two blocks while the pool gets nothing."

"In the second scenario, where the sel fish pool succeeds in finding a second block, it develops a comfortable lead of two blocks that allow it with some cushion against discoveries by the honest miners. Once the pool reaches this point, it continues to mine at the head of its private branch. It publishes one block from its private branch for every block the others find. Since the sel fish pool is a minority, its lead will eventually reduce to a single block with high probability. At this point, the honest miners are too close, so the pool publishes its private branch. Since the private branch is longer than the public branch by one block, it is adopted by all miners as the main branch, and the pool enjoys the revenue of all its blocks. This brings the system back to a state where there is just a single branch until the pool bifurcates it again."

michael_nielsen··on Ask HN: What are you working on and why is it cool?
Python. I wanted to make the underlying ideas as transparent as possible, so I chose a language which many people will find accessible, and I've tried to use simple, easily understandable idioms. I hope it will also be easily accessible to people who use Python rarely.
michael_nielsen··on Ask HN: What are you working on and why is it cool?
Thanks!

The quantum computing book holds up okay, so far as I know. Of course, there's been many new developments since we wrote it, and a text written today would be somewhat different. But the core material in the field has changed only slowly, so a book can be surprisingly durable.

michael_nielsen··on Ask HN: What are you working on and why is it cool?
A book that explains the core ideas of neural networks and deep learning. Cool because:

* The book incorporates lots of running code for readers to explore and extend.

* The book's philosophy is to go deep into the core concepts of deep learning, not to superficially cover a long laundry list of ideas. This gives readers a solid foundation to build on, and makes understanding other material much easier.

* Deep learning is the most powerful approach known to many problems in image recognition, speech recognition, and natural language. The book will help lots more people get quickly up to speed.

The book will be freely available online, and a beta site is coming soon. Pre-beta mailing list here: http://eepurl.com/BYr9L

michael_nielsen··on “I Quit Academia,” an Important, Growing Subgenre of American Essays
Whether it's "quite common" may vary a lot from department to department; in some places my statement may perhaps have been too strong. In the places I've seen firsthand it's quite common for tenured faculty to sometimes interview elsewhere. I did so, and made a jump.

This is also a common strategy for people angling for promotion or a big pay increase -- get an offer elsewhere, and then use that offer to get a promotion at your current institution. I'm not recommending it, but I've certainly seen it done. Common outside academia too, of course.

michael_nielsen··on “I Quit Academia,” an Important, Growing Subgenre of American Essays
It's quite common to move between tenured jobs. Typically you don't have to go through the whole six-year pre-tenure odyssey again. Applying for a tenured job is still quite a process, but that's often true for positions outside academia too, especially senior professional positions.
michael_nielsen··on Americans Fall Behind In The 'Getting Older' Race
No, I mean 1/125. Yes, it is awfully high.
michael_nielsen··on Americans Fall Behind In The 'Getting Older' Race
A little research (sources at end) shows that the many commenters pointing to vehicular accidents, homicide, infant mortality, and suicide as the "reason" for lower US life expectancy are flat-out wrong:

Each year the chance of someone dying in a vehicular accident is about 1 in 10,000 in the US. Over 80 years that means a citizen has roughly a 1 / 125 chance of dying in a vehicular accident. The total reduction in life expectancy from vehicular accidents is thus well under 1.0 years. Assume, for example that such an accident cuts an average of 60 years off a person's life -- they would have lived to 80, say, but their life was cut short at just 20 years old. That works out as a 0.5 year reduction in life expectancy due to accidents.

This is without taking into account the vehicular fatality rates in other countries. When you do that for, e.g., Japan (1 in 20,000 per year), the difference in life expectancy is likely around 0.25 years.

The homicide rate in the US is about 1 in 20,000 people per year, or about half the vehicular fatality rate, so perhaps a further 0.25 year reduction, without factoring in homicides in other countries.

For infant mortality, the chance of dying at birth is a little over 1 in 200 in the US. That translates to a reduction in life expectancy of about 0.4 years, without factoring in infant mortality in other countries.

A similar analysis shows suicide rates are probably not a factor -- in fact, in some countries, such as Japan, the rate is higher than in the US, while in countries such as the UK it is comparable to the US.

So the contributions from all four of these sources is very unlikely to be more than a 1 year difference in life expectancy. It's probably quite a bit less, since I've ignored (e.g.) infant mortality in other countries. Certainly, the contribution from all four of these is nowhere near the dominant effect to explain differences in life expectancy.

A caveat: some people pointing to infant mortality are claiming that what's going on is that a large number of stillbirths simply aren't being reported in some countries outside the US, and this is skewing the results. That possibility isn't accounted for in my analysis above. I doubt it's true - in my many years living in Canada and Australia I've known quite a few people who died in car accidents, and none of whom had a stillbirth (to my knowledge). But while this suggests to me that this effect isn't large, it's of course only anecdotal evidence.

Sources for rates:

http://en.wikipedia.org/wiki/List_of_countries_by_intentiona...

http://en.wikipedia.org/wiki/List_of_countries_by_traffic-re...

http://en.wikipedia.org/wiki/List_of_countries_by_infant_mor...

And if you want to look at the details of the suicide statistics:

http://en.wikipedia.org/wiki/List_of_countries_by_suicide_ra...

michael_nielsen··on What is it like to be a mathematician?
On a related note, there's a good in-depth answer to the question "What is it like to understand advanced mathematics?" here:

http://www.quora.com/Mathematics/What-is-it-like-to-understa...

I'm not a mathematician, but was a theoretical physicist for more than a decade, and the Quora article accords well with my experience in understanding mathematics.

(Fairly) readable examples of some of the points made in the above article may be found in Terry Tao and Scott Aaronson's comments about how to think intuitively in high dimensions:

http://mathoverflow.net/questions/25983/intuitive-crutches-f...

← PreviousPage 5 of 17Next →