The Multi-Armed Bandit Problem

Eugh, this website has custom behavior for the left-right arrow keys, which breaks the back keyboard shortcut in Firefox (alt+left arrow key). Non-standard navigation on a page that IMHO doesn't need it.

Just had a look at your book, and it looks amazing! Hope you will complete it:D

On a separate note, im surprised that in the many discussion of the MAB problem on HN, the Gittins index was never mentioned. Under certain (mild) assumptions on reward distributions, the GI gives a provably optimal solution to the MAB problem...

One thing to expand upon: This is just the classic formulation of the Multi-Armed Bandit Problem. There are countless variations to the formula which have slightly different underlying assumptions.

Good article.

I'd like to know what he used to output those graphs.