A NEURAL TEXT-TO-SPEECH MODEL
UTILIZING BROADCAST DATA MIXED WITH BACKGROUND MUSIC

Hanbin Bae, Jae-Sung Bae, Young-Sun Joo, et al

bhb0722@ncsoft.com

Abstract

    Recently, it has become easier to obtain speech data from various media such as the internet
or YouTube, but directly utilizing them to train a neural text-to-speech (TTS) model is difficult.
The proportion of clean speech is insufficient and the remainder includes background music.
Even with the global style token (GST). Therefore, we propose the following method to successfully
train an end-to-end TTS model with limited broadcast data. First, the background music is removed
from the speech by introducing a music filter. Second, the GST-TTS model with an auxiliary quality
classifier is trained with the filtered speech and a small amount of clean speech. In particular, the
quality classifier makes the embedding vector of the GST layer focus on representing the speech
quality (filtered or clean) of the input speech. The experimental results verified that the proposed
method synthesized much more high-quality speech than conventional methods.

Contents

1. Pre-trained Music Filter
1. KSponSpeech Dataset
2. Comparing music-mixed speech and filtered speech


2. Proposed Systems (models w/ SER lower than 50%)

1. TTS: the DC-TTS [2] model usedconducted in the pre-liminary experiments.
2. GST: the GST-TTS model where the quality embeddingfrom the GST layers was concatenated into the encoderstate of DC-TTS.
3. GST+MF: the GST-TTS model trained with filteredspeech obtained from the pre-trained music filter.
4. GST+MF+Aux.: the GST+MF model with an auxiliaryclassifier.

Sample #1 : 눈은 내일 새벽까지 이어지다가 점차 그치겠고요.

Sample #2 : 도심형 생활주택 구매 어떤가요?

Sample #3 : 또 동해안을 중심으로는 건조 주의보가 발효 중입니다.

Audio Samples