Contextual bandits with continuous actions: Smoothing, zooming, and adapting

Akshay Krishnamurthy; John Langford; Aleksandrs Slivkins; Chicheng Zhang

Contextual bandits with continuous actions: Smoothing, zooming, and adapting

Akshay Krishnamurthy, John Langford, Aleksandrs Slivkins, Chicheng Zhang

Research output: Contribution to journal › Conference article › peer-review

Abstract

We study contextual bandit learning for any competitor policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-dependent “zooming" behavior and, with no tuning, yield improved guarantees for benign problems. We also study adapting to unknown smoothness parameters, establishing a price-of-adaptivity and deriving optimal adaptive algorithms that require no additional information.

Original language	English (US)
Pages (from-to)	2025-2027
Number of pages	3
Journal	Proceedings of Machine Learning Research
Volume	99
State	Published - 2019
Externally published	Yes
Event	32nd Conference on Learning Theory, COLT 2019 - Phoenix, United States Duration: Jun 25 2019 → Jun 28 2019

Keywords

Contextual bandits
Lipschitz bandits
Nonparametric learning

ASJC Scopus subject areas

Software
Control and Systems Engineering
Statistics and Probability
Artificial Intelligence

Cite this

@article{50f72ac713254b6d9f6bff31b9bd1c68,

title = "Contextual bandits with continuous actions: Smoothing, zooming, and adapting",

abstract = "We study contextual bandit learning for any competitor policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-dependent “zooming{"} behavior and, with no tuning, yield improved guarantees for benign problems. We also study adapting to unknown smoothness parameters, establishing a price-of-adaptivity and deriving optimal adaptive algorithms that require no additional information.",

keywords = "Contextual bandits, Lipschitz bandits, Nonparametric learning",

author = "Akshay Krishnamurthy and John Langford and Aleksandrs Slivkins and Chicheng Zhang",

note = "Publisher Copyright: {\textcopyright} 2019 A. Krishnamurthy, J. Langford, A. Slivkins \& C. Zhang.; 32nd Conference on Learning Theory, COLT 2019 ; Conference date: 25-06-2019 Through 28-06-2019",

year = "2019",

language = "English (US)",

volume = "99",

pages = "2025--2027",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

publisher = "ML Research Press",

}

TY - JOUR

T1 - Contextual bandits with continuous actions

T2 - 32nd Conference on Learning Theory, COLT 2019

AU - Krishnamurthy, Akshay

AU - Langford, John

AU - Slivkins, Aleksandrs

AU - Zhang, Chicheng

PY - 2019

Y1 - 2019

N2 - We study contextual bandit learning for any competitor policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-dependent “zooming" behavior and, with no tuning, yield improved guarantees for benign problems. We also study adapting to unknown smoothness parameters, establishing a price-of-adaptivity and deriving optimal adaptive algorithms that require no additional information.

AB - We study contextual bandit learning for any competitor policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-dependent “zooming" behavior and, with no tuning, yield improved guarantees for benign problems. We also study adapting to unknown smoothness parameters, establishing a price-of-adaptivity and deriving optimal adaptive algorithms that require no additional information.

KW - Contextual bandits

KW - Lipschitz bandits

KW - Nonparametric learning

UR - https://www.scopus.com/pages/publications/85160820161

UR - https://www.scopus.com/inward/citedby.url?scp=85160820161&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85160820161

SN - 2640-3498

VL - 99

SP - 2025

EP - 2027

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

Y2 - 25 June 2019 through 28 June 2019

ER -

Contextual bandits with continuous actions: Smoothing, zooming, and adapting

Abstract

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this