[…] the discounted value (between 0 and 1) of the next pull of a bandit arm, taking into account the results of all pulls of that arm to date. The calculation of the index is a bit involved (it involves finding the present value of infinite uncertain sequences), but here are a few values taken from Gittens? 1979 paper introducing the index:
|
wins |
||||
|
losses |
1 |
2 |
3 |
|
|
1 |
.6211 |
.7465 |
.8062 |
|
|
2 |
.4256 |
.5760 |
.6607 |
|
|
3 |
.3182 |
.4641 |
.5554 |
|
Table of Gittens index values for given numbers of wins and losses
For example, given a discount rate of 75%, and given that you have pulled the arm 3 times and won twice, the Gittens index is 0.7465, meaning that continuing to pull for an indefinite period is the equivalent of a certain immediate payout of 0.7465. Note that for all the 50/50 cells where wins = losses, despite the fact that the data appear to be suggesting parity between wins and losses, the Gittens index is > 0.5. This reflects the hidden value in the ability to continue to explore.