Bandit Problem Resources
This is a survey paper that describes bandit problems, gives several examples from economics and OR, and discusses general techniques. It's a good place to get started and we will certainly look at some of the "story" problems.
This paper does a great job putting the various approaches to the Gittins index theorem and its variations. We'll surely follow this path in class.
Incidentally, this is a great model for a survey paper. It does not try to cover every variation or every application; it just focuses on giving a clear and complete coverage of the PROOFs of the index theorem. I've used (or tried to use) this kind of model a couple of times: once for Poisson Approximation and once for Monotone Subsquence.
This is more demanding mathematically, but it puts you honestly onto the research trail. We probably won't go far into this paper. I may summarize some bits.
The title spills the beans. A succinct induction proof is obtained and computations are minimized. Still, for learning what is going on here, you will do well to look first at the exposition of Frostig and Weiss.