WaveGlowGAN: the bipartite flow based vocoder with generative adversarial networks for high quality speech synthesis

Authors

Gyeong-Hoon Lee, Junmo Lee, Young-Ik Kim, Hoon-Young Cho (NCSOFT)

Abstract

In this study, we propose an effective learning method to improve the quality of speech synthesized by WaveGlow. WaveGlow is a recently proposed neural vocoder for parallel speech generation based on a bipartite flow. Because the transformation in WaveGlow is invertible, the network (a) is trained to maximize the log-likelihood of true audio datasets in the forward direction and (b) generates speech from a sample in the simple known distribution in the backward direction. WaveGlow generates high-quality speech from a mel-spectrogram faster than autoregressive neural vocoders. However, even if the log-likelihood loss converges sufficiently, the quality of speech generated by WaveGlow is not significantly improved compared with other generative models trained by the regression or adversarial loss, which directly compare the output sample with true data. In this paper, we propose a novel learning method for WaveGlow by combining the log-likelihood and adversarial loss. In our method, we train the network of WaveGlow in the forward and backward direction alternately by using invertible mappings of WaveGlow. In the forward direction, as in the conventional method, we train the network to maximize the log-likelihood of the training dataset. In the backward direction, we train the network to minimimze the loss between the spectrograms of the synthesized and real audio samples using the $L1$ loss and adversarial loss. By applying the reconstruction loss to the generative model that has a bipartite flow, the distortion and noise of generated speech are significantly reduced compared with using conventional learning alone. The experimental results prove that our method is an effective way to train the network of WaveGlow.