Abstract
The loss function expresses the differences between the predicted values from regression or classification models and the actual instances in machine learning. Regularization also plays an important role in machine learning, and it can mitigate overfitting problems, perform variable selection, and produce sparse models. The hyperparameter in these models controls the trade-off between the loss function and the regularization term, as well as the bias-variance trade-off. The choice of hyperparameter will influence the performance of the models. Thus, the hyperparameter needs to be tuned for effective learning from data. In some machine learning models, the optimal values for estimated coefficients are piecewise linear with respect to the hyperparameter. Efficient algorithms can be developed to compute all solutions, and these kinds of methods are called solution path algorithms. They can significantly reduce the efforts for cross-validation and highly speed up hyperparameter tuning. In this paper, we review the solution path algorithms widely used in regression and classification machine learning problems.
Original language | English (US) |
---|---|
Pages (from-to) | 749-789 |
Number of pages | 41 |
Journal | Annals of Data Science |
Volume | 9 |
Issue number | 4 |
DOIs | |
State | Published - Aug 2022 |
Keywords
- Classification
- Machine learning
- Regression
- Regularization
- Solution path
ASJC Scopus subject areas
- Business, Management and Accounting (miscellaneous)
- Computer Science Applications
- Statistics, Probability and Uncertainty
- Artificial Intelligence