Thanks for the comment. The third blog post in this series will discuss linear contextual bandits, for which I used Langford's VW.

It does a huge amount better.