The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Dans cet article, nous étudions analytiquement les performances de généralisation de l'apprentissage en utilisant des entrées corrélées dans le cadre de l'apprentissage en ligne avec une méthode mécanique statistique. Nous considérons un modèle composé de perceptrons linéaires avec bruit gaussien. Dans un premier temps, nous analysons le cas de la méthode du gradient. Nous clarifions analytiquement que plus la corrélation entre les entrées est grande ou plus le nombre d'entrées est grand, plus la condition que le taux d'apprentissage doit satisfaire est stricte et plus la vitesse d'apprentissage est lente. Deuxièmement, nous traitons l’apprentissage par projection orthogonale par bloc comme une règle d’apprentissage alternative et en déduisons la théorie. Dans un cas silencieux, la vitesse d'apprentissage ne dépend pas de la corrélation et est proportionnelle au nombre d'entrées utilisées dans une mise à jour. La vitesse d'apprentissage est identique à celle de la méthode du gradient avec entrées non corrélées. D’un autre côté, lorsqu’il y a du bruit, plus la corrélation entre les entrées est grande, plus la vitesse d’apprentissage est lente et plus l’erreur de généralisation résiduelle est grande.
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Chihiro SEKI, Shingo SAKURAI, Masafumi MATSUNO, Seiji MIYOSHI, "A Theoretical Analysis of On-Line Learning Using Correlated Examples" in IEICE TRANSACTIONS on Fundamentals,
vol. E91-A, no. 9, pp. 2663-2670, September 2008, doi: 10.1093/ietfec/e91-a.9.2663.
Abstract: In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1093/ietfec/e91-a.9.2663/_p
Copier
@ARTICLE{e91-a_9_2663,
author={Chihiro SEKI, Shingo SAKURAI, Masafumi MATSUNO, Seiji MIYOSHI, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={A Theoretical Analysis of On-Line Learning Using Correlated Examples},
year={2008},
volume={E91-A},
number={9},
pages={2663-2670},
abstract={In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.},
keywords={},
doi={10.1093/ietfec/e91-a.9.2663},
ISSN={1745-1337},
month={September},}
Copier
TY - JOUR
TI - A Theoretical Analysis of On-Line Learning Using Correlated Examples
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 2663
EP - 2670
AU - Chihiro SEKI
AU - Shingo SAKURAI
AU - Masafumi MATSUNO
AU - Seiji MIYOSHI
PY - 2008
DO - 10.1093/ietfec/e91-a.9.2663
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E91-A
IS - 9
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - September 2008
AB - In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
ER -