The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Cet article présente une méthode d'élagage, Reconstruction Error Aware Pruning (REAP), pour réduire la redondance des modèles de réseaux neuronaux convolutifs afin d'accélérer leur inférence. Dans REAP, nous avons les étapes suivantes : 1) Élaguer les canaux dont les sorties sont redondantes et peuvent être reconstruites à partir des sorties des autres canaux dans chaque couche convolutive ; 2) Mettre à jour les poids des canaux restants par la méthode des moindres carrés afin de compenser l'erreur provoquée par l'élagage. C'est ainsi que nous compressons et accélérons les modèles initialement volumineux et lents avec peu de dégradation. La capacité de REAP à maintenir les performances du modèle nous fait gagner beaucoup de temps et de travail pour recycler les modèles élagués. Le défi de REAP réside dans le coût de calcul nécessaire à la sélection des canaux à élaguer. Pour sélectionner les canaux, nous devons résoudre un grand nombre de problèmes de moindres carrés. Nous avons développé un algorithme efficace basé sur le système biorthogonal pour obtenir les solutions de ces problèmes des moindres carrés. Dans les expériences, nous montrons que REAP peut effectuer un élagage avec un moindre sacrifice des performances du modèle que plusieurs méthodes existantes, y compris celle qui était auparavant à la pointe de la technologie.
Koji KAMMA
Wakayama University
Toshikazu WADA
Wakayama University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Koji KAMMA, Toshikazu WADA, "REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation" in IEICE TRANSACTIONS on Information,
vol. E104-D, no. 1, pp. 194-202, January 2021, doi: 10.1587/transinf.2020EDP7049.
Abstract: This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2020EDP7049/_p
Copier
@ARTICLE{e104-d_1_194,
author={Koji KAMMA, Toshikazu WADA, },
journal={IEICE TRANSACTIONS on Information},
title={REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation},
year={2021},
volume={E104-D},
number={1},
pages={194-202},
abstract={This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.},
keywords={},
doi={10.1587/transinf.2020EDP7049},
ISSN={1745-1361},
month={January},}
Copier
TY - JOUR
TI - REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation
T2 - IEICE TRANSACTIONS on Information
SP - 194
EP - 202
AU - Koji KAMMA
AU - Toshikazu WADA
PY - 2021
DO - 10.1587/transinf.2020EDP7049
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E104-D
IS - 1
JA - IEICE TRANSACTIONS on Information
Y1 - January 2021
AB - This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.
ER -