A Sequential Learning Approach for Scaling Up Filter-Based Feature Subset Selection

Gregory Ditzler, Robi Polikar, Gail Rosen

Research output: Contribution to journalArticlepeer-review

18 Scopus citations

Abstract

Increasingly, many machine learning applications are now associated with very large data sets whose sizes were almost unimaginable just a short time ago. As a result, many of the current algorithms cannot handle, or do not scale to, today's extremely large volumes of data. Fortunately, not all features that make up a typical data set carry information that is relevant or useful for prediction, and identifying and removing such irrelevant features can significantly reduce the total data size. The unfortunate dilemma, however, is that some of the current data sets are so large that common feature selection algorithms - whose very goal is to reduce the dimensionality - cannot handle such large data sets, creating a vicious cycle. We describe a sequential learning framework for feature subset selection (SLSS) that can scale with both the number of features and the number of observations. The proposed framework uses multiarm bandit algorithms to sequentially search a subset of variables, and assign a level of importance for each feature. The novel contribution of SLSS is its ability to naturally scale to large data sets, evaluate such data in a very small amount of time, and be performed independently of the optimization of any classifier to reduce unnecessary complexity. We demonstrate the capabilities of SLSS on synthetic and real-world data sets.

Original languageEnglish (US)
Pages (from-to)2530-2544
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number6
DOIs
StatePublished - Jun 2018

Keywords

  • Feature selection
  • multiarm bandit
  • sequential learning
  • subset selection

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'A Sequential Learning Approach for Scaling Up Filter-Based Feature Subset Selection'. Together they form a unique fingerprint.

Cite this