N-Singer2: Improving the naturalness of synthesized singing voice

in the Korean pop ballad genre

Gyeong-Hoon Lee, Tae-Woo Kim, Minsu Kang

{ghlee3401, ktw0114, mskang}@ncsoft.com

Abstract

 Singing voices in the pop ballad genre especially tend to have more dynamic and complicated components
related to vocal techniques of a singer, such as vibrato, breathy sound, and fundamental frequency (F0) fluctuations,
than the voices suited to the children’s song genre. It is important to understand the characteristics of a singer
to generate a natural singing voice. However, it is difficult to model a singing voice for pop ballads
because both linguistic content and singer identity are entangled with prosody information.
In this paper, we propose modeling natural singing voices by disentangling singer-independent
and singer-dependent features using a multi-singer system. In the variation predictor of our system,
for each input note pitch, we use a difference-based F0 to model the vibrato and F0 fluctuations
only related to the characteristics of the singer. Our system also predict the voiced range of a singing voice to prevent
predicting pitch in the unvoiced range arising from the wrong duration of musical notes. In addition, we apply a data
augmentation method to the training dataset for generating a singing voice with a more stable prosody.
Experimental results show that our system can generate a more natural singing voice than the conventional system.

Structure

Contents

1. Audio Samples (Korean ballad song genre)
2. Audio Samples (Korean children’s song genre)