The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Cet article aborde le problème de la sélection d'un sous-ensemble significatif de caractéristiques candidates à utiliser pour la régression linéaire multiple. Bertsimas et al. [5] ont récemment proposé l'algorithme discret du premier ordre (DFO) pour trouver efficacement des solutions quasi optimales à ce problème. Cependant, cet algorithme ne peut échapper aux solutions localement optimales. Pour résoudre ce problème, nous proposons un algorithme stochastique discret de premier ordre (SDFO) pour la sélection de sous-ensembles de fonctionnalités. Dans cet algorithme, des perturbations aléatoires sont ajoutées à une séquence de solutions candidates afin d'échapper aux solutions localement optimales, ce qui élargit la gamme de solutions détectables. De plus, nous dérivons la taille de pas optimale dans la direction de descente du gradient pour accélérer la convergence de l'algorithme. Nous utilisons également efficacement les L2-terme de régularisation pour améliorer les performances prédictives d'un modèle de régression de sous-ensemble résultant. Les résultats de la simulation démontrent que notre algorithme surpasse considérablement l'algorithme original du MPO. Notre algorithme était également supérieur en termes de performances prédictives au lasso et à la sélection étape par étape.
Kota KUDO
University of Tsukuba
Yuichi TAKANO
University of Tsukuba
Ryo NOMURA
Waseda University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Kota KUDO, Yuichi TAKANO, Ryo NOMURA, "Stochastic Discrete First-Order Algorithm for Feature Subset Selection" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 7, pp. 1693-1702, July 2020, doi: 10.1587/transinf.2019EDP7274.
Abstract: This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7274/_p
Copier
@ARTICLE{e103-d_7_1693,
author={Kota KUDO, Yuichi TAKANO, Ryo NOMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Stochastic Discrete First-Order Algorithm for Feature Subset Selection},
year={2020},
volume={E103-D},
number={7},
pages={1693-1702},
abstract={This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.},
keywords={},
doi={10.1587/transinf.2019EDP7274},
ISSN={1745-1361},
month={July},}
Copier
TY - JOUR
TI - Stochastic Discrete First-Order Algorithm for Feature Subset Selection
T2 - IEICE TRANSACTIONS on Information
SP - 1693
EP - 1702
AU - Kota KUDO
AU - Yuichi TAKANO
AU - Ryo NOMURA
PY - 2020
DO - 10.1587/transinf.2019EDP7274
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2020
AB - This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
ER -