Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

Chicheng Zhang; Alekh Agarwal; Hal Daumé; John Langford; Sahand N. Negahban

Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

Chicheng Zhang
, Alekh Agarwal
, Hal Daumé
, John Langford
, Sahand N. Negahban

Research output: Contribution to journal › Conference article › peer-review

Abstract

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

Original language	English (US)
Pages (from-to)	7335-7344
Number of pages	10
Journal	Proceedings of Machine Learning Research
Volume	97
State	Published - 2019
Externally published	Yes
Event	36th International Conference on Machine Learning, ICML 2019 - Long Beach, United States Duration: Jun 9 2019 → Jun 15 2019

ASJC Scopus subject areas

Software
Control and Systems Engineering
Statistics and Probability
Artificial Intelligence

Cite this

@article{c36bff15dacd4e3ca63c323ddb99f753,

title = "Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback",

abstract = "We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.",

author = "Chicheng Zhang and Alekh Agarwal and Hal Daum{\'e} and John Langford and Negahban, \{Sahand N.\}",

note = "Publisher Copyright: {\textcopyright} 2019 by the author(s).; 36th International Conference on Machine Learning, ICML 2019 ; Conference date: 09-06-2019 Through 15-06-2019",

year = "2019",

language = "English (US)",

volume = "97",

pages = "7335--7344",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

publisher = "ML Research Press",

}

TY - JOUR

T1 - Warm-starting Contextual Bandits

T2 - 36th International Conference on Machine Learning, ICML 2019

AU - Zhang, Chicheng

AU - Agarwal, Alekh

AU - Daumé, Hal

AU - Langford, John

AU - Negahban, Sahand N.

PY - 2019

Y1 - 2019

N2 - We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

AB - We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible, and helpful in practice.

UR - https://www.scopus.com/pages/publications/105020258371

UR - https://www.scopus.com/pages/publications/105020258371#tab=citedBy

M3 - Conference article

AN - SCOPUS:105020258371

SN - 2640-3498

VL - 97

SP - 7335

EP - 7344

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

Y2 - 9 June 2019 through 15 June 2019

ER -

Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

Abstract

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this