Robust model-free multiclass probability estimation

Yichao Wu, Hao Helen Zhang, Yufeng Liu

Research output: Contribution to journalArticlepeer-review

28 Scopus citations

Abstract

Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.

Original languageEnglish (US)
Pages (from-to)424-436
Number of pages13
JournalJournal of the American Statistical Association
Volume105
Issue number489
DOIs
StatePublished - Mar 2010
Externally publishedYes

Keywords

  • Fisher consistency
  • Hard classification
  • Multicategory classification
  • Probability estimation
  • SVM
  • Soft classification

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'Robust model-free multiclass probability estimation'. Together they form a unique fingerprint.

Cite this