Warm-starting contextual bandits: Robustly combining supervised and bandit feedback

Chicheng Zhang; Alekh Agarwal; Hal Daumé; John Langford; Sahand N. Negahban

Warm-starting contextual bandits: Robustly combining supervised and bandit feedback

Chicheng Zhang, Alekh Agarwal, Hal Daumé, John Langford, Sahand N. Negahban

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

Original language	English (US)
Title of host publication	36th International Conference on Machine Learning, ICML 2019
Publisher	International Machine Learning Society (IMLS)
Pages	12723-12732
Number of pages	10
ISBN (Electronic)	9781510886988
State	Published - 2019
Externally published	Yes
Event	36th International Conference on Machine Learning, ICML 2019 - Long Beach, United States Duration: Jun 9 2019 → Jun 15 2019

Publication series

Name	36th International Conference on Machine Learning, ICML 2019
Volume	2019-June

Conference

Conference	36th International Conference on Machine Learning, ICML 2019
Country/Territory	United States
City	Long Beach
Period	6/9/19 → 6/15/19

ASJC Scopus subject areas

Education
Computer Science Applications
Human-Computer Interaction

Cite this

Zhang, C., Agarwal, A., Daumé, H., Langford, J., & Negahban, S. N. (2019). Warm-starting contextual bandits: Robustly combining supervised and bandit feedback. In 36th International Conference on Machine Learning, ICML 2019 (pp. 12723-12732). (36th International Conference on Machine Learning, ICML 2019; Vol. 2019-June). International Machine Learning Society (IMLS).

Warm-starting contextual bandits: Robustly combining supervised and bandit feedback. / Zhang, Chicheng; Agarwal, Alekh; Daumé, Hal et al.
36th International Conference on Machine Learning, ICML 2019. International Machine Learning Society (IMLS), 2019. p. 12723-12732 (36th International Conference on Machine Learning, ICML 2019; Vol. 2019-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zhang, C, Agarwal, A, Daumé, H, Langford, J & Negahban, SN 2019, Warm-starting contextual bandits: Robustly combining supervised and bandit feedback. in 36th International Conference on Machine Learning, ICML 2019. 36th International Conference on Machine Learning, ICML 2019, vol. 2019-June, International Machine Learning Society (IMLS), pp. 12723-12732, 36th International Conference on Machine Learning, ICML 2019, Long Beach, United States, 6/9/19.

@inproceedings{020629f2f8624d70ba02246345f8a8db,

title = "Warm-starting contextual bandits: Robustly combining supervised and bandit feedback",

abstract = "We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.",

author = "Chicheng Zhang and Alekh Agarwal and Hal Daum{\'e} and John Langford and Negahban, {Sahand N.}",

note = "Publisher Copyright: {\textcopyright} 36th International Conference on Machine Learning, ICML 2019. All rights reserved.; 36th International Conference on Machine Learning, ICML 2019 ; Conference date: 09-06-2019 Through 15-06-2019",

year = "2019",

language = "English (US)",

series = "36th International Conference on Machine Learning, ICML 2019",

publisher = "International Machine Learning Society (IMLS)",

pages = "12723--12732",

booktitle = "36th International Conference on Machine Learning, ICML 2019",

}

TY - GEN

T1 - Warm-starting contextual bandits

T2 - 36th International Conference on Machine Learning, ICML 2019

AU - Zhang, Chicheng

AU - Agarwal, Alekh

AU - Daumé, Hal

AU - Langford, John

AU - Negahban, Sahand N.

PY - 2019

Y1 - 2019

N2 - We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

AB - We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

UR - http://www.scopus.com/inward/record.url?scp=85078052153&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85078052153&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85078052153

T3 - 36th International Conference on Machine Learning, ICML 2019

SP - 12723

EP - 12732

BT - 36th International Conference on Machine Learning, ICML 2019

PB - International Machine Learning Society (IMLS)

Y2 - 9 June 2019 through 15 June 2019

ER -

Warm-starting contextual bandits: Robustly combining supervised and bandit feedback

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this