Controlled semi-Markov models under long-run average rewards

Rabi N. Bhattacharya, Mukul Majumdar

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved and a dynamic programming equation derived for general semi-Markov models under the long-run average reward criterion, focusing on it as a limiting case of optimization under discounting as the discount factor goes to one. This extends many earlier results. An example of Reed (1974) on harvesting a natural resource provides an application not covered by earlier results.

Original languageEnglish (US)
Pages (from-to)223-242
Number of pages20
JournalJournal of Statistical Planning and Inference
Issue number2
StatePublished - Jun 1989
Externally publishedYes


  • Stationary optimal policy
  • dynamic programming equation
  • equicontinuity
  • uniformity in weak convergence

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty
  • Applied Mathematics


Dive into the research topics of 'Controlled semi-Markov models under long-run average rewards'. Together they form a unique fingerprint.

Cite this