The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Les réseaux contradictoires génératifs (GAN) sont l'un des principes d'apprentissage les plus efficaces des modèles génératifs et ont été largement appliqués à de nombreuses tâches de génération. Au début, la pénalité de gradient (GP) a été appliquée pour appliquer le discriminateur dans les GAN afin de satisfaire la continuité Lipschitzienne dans le GAN de Wasserstein. Bien que la version standard de la pénalité de gradient ait été encore modifiée à des fins différentes, la recherche d’un meilleur équilibre et d’une qualité de génération supérieure dans l’apprentissage contradictoire reste un défi. Récemment, DRAGAN a été proposé pour atteindre la linéarité locale dans une variété de données environnante en appliquant la pénalité de gradient bruité pour promouvoir la convexité locale dans l'optimisation du modèle. Cependant, nous montrons que leur approche imposera un fardeau pour satisfaire la continuité lipschitzienne pour le discriminateur. Un tel conflit entre la continuité lipschitzienne et la linéarité locale dans DRAGAN entraînera un mauvais équilibre et la qualité de la génération est donc loin d'être idéale. À cette fin, nous proposons une nouvelle approche qui profite à la fois à la linéarité locale et à la continuité lipschitzienne pour atteindre un meilleur équilibre sans conflit. En détail, nous appliquons notre fonction d'activation synchronisée dans le discriminateur pour recevoir une forme particulière de pénalité de gradient bruité afin d'atteindre la linéarité locale sans perdre la propriété de continuité Lipschitzienne dans le discriminateur. Les résultats expérimentaux montrent que notre méthode peut atteindre la qualité supérieure des images et surpasse WGAN-GP, DiracGAN et DRAGAN en termes de score de début et de distance de début de Fréchet sur des ensembles de données du monde réel.
Rui YANG
University of Tokyo
Raphael SHU
Amazon AI
Hideki NAKAYAMA
University of Tokyo
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copier
Rui YANG, Raphael SHU, Hideki NAKAYAMA, "Improving Noised Gradient Penalty with Synchronized Activation Function for Generative Adversarial Networks" in IEICE TRANSACTIONS on Information,
vol. E105-D, no. 9, pp. 1537-1545, September 2022, doi: 10.1587/transinf.2022EDP7019.
Abstract: Generative Adversarial Networks (GANs) are one of the most successful learning principles of generative models and were wildly applied to many generation tasks. In the beginning, the gradient penalty (GP) was applied to enforce the discriminator in GANs to satisfy Lipschitz continuity in Wasserstein GAN. Although the vanilla version of the gradient penalty was further modified for different purposes, seeking a better equilibrium and higher generation quality in adversarial learning remains challenging. Recently, DRAGAN was proposed to achieve the local linearity in a surrounding data manifold by applying the noised gradient penalty to promote the local convexity in model optimization. However, we show that their approach will impose a burden on satisfying Lipschitz continuity for the discriminator. Such conflict between Lipschitz continuity and local linearity in DRAGAN will result in poor equilibrium, and thus the generation quality is far from ideal. To this end, we propose a novel approach to benefit both local linearity and Lipschitz continuity for reaching a better equilibrium without conflict. In detail, we apply our synchronized activation function in the discriminator to receive a particular form of noised gradient penalty for achieving local linearity without losing the property of Lipschitz continuity in the discriminator. Experimental results show that our method can reach the superior quality of images and outperforms WGAN-GP, DiracGAN, and DRAGAN in terms of Inception Score and Fréchet Inception Distance on real-world datasets.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7019/_p
Copier
@ARTICLE{e105-d_9_1537,
author={Rui YANG, Raphael SHU, Hideki NAKAYAMA, },
journal={IEICE TRANSACTIONS on Information},
title={Improving Noised Gradient Penalty with Synchronized Activation Function for Generative Adversarial Networks},
year={2022},
volume={E105-D},
number={9},
pages={1537-1545},
abstract={Generative Adversarial Networks (GANs) are one of the most successful learning principles of generative models and were wildly applied to many generation tasks. In the beginning, the gradient penalty (GP) was applied to enforce the discriminator in GANs to satisfy Lipschitz continuity in Wasserstein GAN. Although the vanilla version of the gradient penalty was further modified for different purposes, seeking a better equilibrium and higher generation quality in adversarial learning remains challenging. Recently, DRAGAN was proposed to achieve the local linearity in a surrounding data manifold by applying the noised gradient penalty to promote the local convexity in model optimization. However, we show that their approach will impose a burden on satisfying Lipschitz continuity for the discriminator. Such conflict between Lipschitz continuity and local linearity in DRAGAN will result in poor equilibrium, and thus the generation quality is far from ideal. To this end, we propose a novel approach to benefit both local linearity and Lipschitz continuity for reaching a better equilibrium without conflict. In detail, we apply our synchronized activation function in the discriminator to receive a particular form of noised gradient penalty for achieving local linearity without losing the property of Lipschitz continuity in the discriminator. Experimental results show that our method can reach the superior quality of images and outperforms WGAN-GP, DiracGAN, and DRAGAN in terms of Inception Score and Fréchet Inception Distance on real-world datasets.},
keywords={},
doi={10.1587/transinf.2022EDP7019},
ISSN={1745-1361},
month={September},}
Copier
TY - JOUR
TI - Improving Noised Gradient Penalty with Synchronized Activation Function for Generative Adversarial Networks
T2 - IEICE TRANSACTIONS on Information
SP - 1537
EP - 1545
AU - Rui YANG
AU - Raphael SHU
AU - Hideki NAKAYAMA
PY - 2022
DO - 10.1587/transinf.2022EDP7019
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E105-D
IS - 9
JA - IEICE TRANSACTIONS on Information
Y1 - September 2022
AB - Generative Adversarial Networks (GANs) are one of the most successful learning principles of generative models and were wildly applied to many generation tasks. In the beginning, the gradient penalty (GP) was applied to enforce the discriminator in GANs to satisfy Lipschitz continuity in Wasserstein GAN. Although the vanilla version of the gradient penalty was further modified for different purposes, seeking a better equilibrium and higher generation quality in adversarial learning remains challenging. Recently, DRAGAN was proposed to achieve the local linearity in a surrounding data manifold by applying the noised gradient penalty to promote the local convexity in model optimization. However, we show that their approach will impose a burden on satisfying Lipschitz continuity for the discriminator. Such conflict between Lipschitz continuity and local linearity in DRAGAN will result in poor equilibrium, and thus the generation quality is far from ideal. To this end, we propose a novel approach to benefit both local linearity and Lipschitz continuity for reaching a better equilibrium without conflict. In detail, we apply our synchronized activation function in the discriminator to receive a particular form of noised gradient penalty for achieving local linearity without losing the property of Lipschitz continuity in the discriminator. Experimental results show that our method can reach the superior quality of images and outperforms WGAN-GP, DiracGAN, and DRAGAN in terms of Inception Score and Fréchet Inception Distance on real-world datasets.
ER -