The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
La descente de gradient naturel est une méthode d'optimisation des réseaux de neurones à valeur réelle proposée du point de vue de la géométrie de l'information. Nous présentons ici une extension de la descente de gradient naturelle vers des réseaux de neurones à valeurs complexes. Notre idée est d’utiliser l’extension hermitienne de la matrice d’information de Fisher. De plus, nous généralisons le gradient naturel projeté (PRONG), qui est un algorithme de descente de gradient naturel rapide, aux réseaux de neurones à valeurs complexes. Nous considérons également l'avantage des réseaux de neurones à valeurs complexes par rapport aux réseaux de neurones à valeurs réelles. Une propriété utile des nombres complexes dans le plan complexe est que la rotation est simplement exprimée par la multiplication. En nous concentrant sur cette propriété, nous construisons la fonction de sortie des réseaux de neurones à valeurs complexes, qui est invariante même si l'entrée est modifiée à sa valeur pivotée. Ensuite, notre réseau neuronal aux valeurs complexes peut apprendre les données pivotées sans augmentation des données. Enfin, grâce à la simulation de la reconnaissance de caractères en ligne, nous démontrons l'efficacité de l'approche proposée.
Jun-ichi MUKUNO
Kogakuin University
Hajime MATSUI
Toyota Technological Institute
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Jun-ichi MUKUNO, Hajime MATSUI, "Natural Gradient Descent of Complex-Valued Neural Networks Invariant under Rotations" in IEICE TRANSACTIONS on Fundamentals,
vol. E102-A, no. 12, pp. 1988-1996, December 2019, doi: 10.1587/transfun.E102.A.1988.
Abstract: The natural gradient descent is an optimization method for real-valued neural networks that was proposed from the viewpoint of information geometry. Here, we present an extension of the natural gradient descent to complex-valued neural networks. Our idea is to use the Hermitian extension of the Fisher information matrix. Moreover, we generalize the projected natural gradient (PRONG), which is a fast natural gradient descent algorithm, to complex-valued neural networks. We also consider the advantage of complex-valued neural networks over real-valued neural networks. A useful property of complex numbers in the complex plane is that the rotation is simply expressed by the multiplication. By focusing on this property, we construct the output function of complex-valued neural networks, which is invariant even if the input is changed to its rotated value. Then, our complex-valued neural network can learn rotated data without data augmentation. Finally, through simulation of online character recognition, we demonstrate the effectiveness of the proposed approach.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.E102.A.1988/_p
Copier
@ARTICLE{e102-a_12_1988,
author={Jun-ichi MUKUNO, Hajime MATSUI, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Natural Gradient Descent of Complex-Valued Neural Networks Invariant under Rotations},
year={2019},
volume={E102-A},
number={12},
pages={1988-1996},
abstract={The natural gradient descent is an optimization method for real-valued neural networks that was proposed from the viewpoint of information geometry. Here, we present an extension of the natural gradient descent to complex-valued neural networks. Our idea is to use the Hermitian extension of the Fisher information matrix. Moreover, we generalize the projected natural gradient (PRONG), which is a fast natural gradient descent algorithm, to complex-valued neural networks. We also consider the advantage of complex-valued neural networks over real-valued neural networks. A useful property of complex numbers in the complex plane is that the rotation is simply expressed by the multiplication. By focusing on this property, we construct the output function of complex-valued neural networks, which is invariant even if the input is changed to its rotated value. Then, our complex-valued neural network can learn rotated data without data augmentation. Finally, through simulation of online character recognition, we demonstrate the effectiveness of the proposed approach.},
keywords={},
doi={10.1587/transfun.E102.A.1988},
ISSN={1745-1337},
month={December},}
Copier
TY - JOUR
TI - Natural Gradient Descent of Complex-Valued Neural Networks Invariant under Rotations
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1988
EP - 1996
AU - Jun-ichi MUKUNO
AU - Hajime MATSUI
PY - 2019
DO - 10.1587/transfun.E102.A.1988
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E102-A
IS - 12
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - December 2019
AB - The natural gradient descent is an optimization method for real-valued neural networks that was proposed from the viewpoint of information geometry. Here, we present an extension of the natural gradient descent to complex-valued neural networks. Our idea is to use the Hermitian extension of the Fisher information matrix. Moreover, we generalize the projected natural gradient (PRONG), which is a fast natural gradient descent algorithm, to complex-valued neural networks. We also consider the advantage of complex-valued neural networks over real-valued neural networks. A useful property of complex numbers in the complex plane is that the rotation is simply expressed by the multiplication. By focusing on this property, we construct the output function of complex-valued neural networks, which is invariant even if the input is changed to its rotated value. Then, our complex-valued neural network can learn rotated data without data augmentation. Finally, through simulation of online character recognition, we demonstrate the effectiveness of the proposed approach.
ER -