TY - JOUR
T1 - Robust model-free multiclass probability estimation
AU - Wu, Yichao
AU - Zhang, Hao Helen
AU - Liu, Yufeng
N1 - Funding Information:
Yichao Wu is Assistant Professor (E-mail: [email protected]), Hao Helen Zhang is Associate Professor (E-mail: [email protected]), Department of Statistics, North Carolina State University, Raleigh, NC 27695. Yufeng Liu is Associate Professor, Department of Statistics and Operations Research, Carolina Center for Genome Sciences, University of North Carolina, Chapel Hill, NC 27599-3260 (E-mail: [email protected]). The authors thank the editor, the associate editor, and two referees for their helpful suggestions that led to significant improvement of the article. The authors are supported in part by NSF grants DMS-0905561 (Wu), DMS-0645293 (Zhang), DMS-0747575 (Liu), and DMS-0606577 (Liu), and NIH/NCI grants R01-CA-085848 (Zhang) and R01-CA-149569 (Liu and Wu).
PY - 2010/3
Y1 - 2010/3
N2 - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.
AB - Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.
KW - Fisher consistency
KW - Hard classification
KW - Multicategory classification
KW - Probability estimation
KW - SVM
KW - Soft classification
UR - http://www.scopus.com/inward/record.url?scp=77952567231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77952567231&partnerID=8YFLogxK
U2 - 10.1198/jasa.2010.tm09107
DO - 10.1198/jasa.2010.tm09107
M3 - Article
AN - SCOPUS:77952567231
SN - 0162-1459
VL - 105
SP - 424
EP - 436
JO - Journal of the American Statistical Association
JF - Journal of the American Statistical Association
IS - 489
ER -