Towards optimization of a human-inspired heuristic for solving explore-exploit problems

Paul Reverdy, Robert C. Wilson, Philip Holmes, Naomi E. Leonard

Research output: Contribution to journalConference articlepeer-review

5 Scopus citations


Motivated by models of human decision making, we consider a heuristic solution for explore-exploit problems. In a numerical example we show that, with appropriate parameter values, the algorithm performs well. However, the parameters of the algorithm trade off exploration against exploitation in a complicated way so that finding the optimal parameter values is not obvious. We show that the optimal parameter values can be analytically computed in some cases and prove that suboptimal parameter tunings can provide robustness to modeling error. The analytic results suggest a feedback control law for dynamically optimizing parameters.

Original languageEnglish (US)
Article number6426148
Pages (from-to)2820-2825
Number of pages6
JournalProceedings of the IEEE Conference on Decision and Control
StatePublished - 2012
Externally publishedYes
Event51st IEEE Conference on Decision and Control, CDC 2012 - Maui, HI, United States
Duration: Dec 10 2012Dec 13 2012

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization


Dive into the research topics of 'Towards optimization of a human-inspired heuristic for solving explore-exploit problems'. Together they form a unique fingerprint.

Cite this