ci_logo

Hierarchical Context-Aware Transformers
for Non-AutoRegressive Text to Speech

Jae-Sung Bae, Tae-Jun Bak, Young-Sun Joo, Hoon-Young Cho

jaesungbae@ncsoft.com


Abstract
    This paper proposes a hierarchical Transformer structure to increase the modeling performance
of Transformer-based non-autoregressive text-to-speech (TNA-TTS). Considering the fact that the
text encoder and the audio decoder of TNA-TTS deal with different types and lengths of data and
hence the contextual information should be dealt with differently at each layer. In this work, we
limited the attention of each layer with different ranges to provide a hierarchical structure. Addit-
ionally, to improve the pitch modeling accuracy of the decoder, we provided sentence- and word-
level pitch as additional conditions. Both objective and subjective evaluations showed that the
proposed approach (hierarchical context-aware Transformers) improved the naturalness, pitch
prediction and intelligibility of the synthesized speech samples from the baseline TNA-TTS.
the predicted pitch.

Contents
  1. Audio Samples (Korean)
Demo page of Hierarchical TNA-TTS


1. Audio Samples (Korean)
Sentence: 난 니가 예전이라길래 한 삼 사 년 전 말하는 줄 알았는데?
(Pronunciation): Nan niga yejeon-ilagillae han sam sa nyeon jeon malhaneun jul al-assneunde?
FastPitch (Baseline) EGW + DW (Ours) EGW + DW + HPC (Ours)
 
Sentence: 그런 것처럼 계속 뭔가 생겨날 거 같아서 차라리 자식을 안 낳는 게 훨씬 더 좋은 거 같아.
(Pronunciation): Geuleon geoscheoleom gyesog mwonga saeng-gyeonal geo gat-aseo chalali jasig-eul an nahneun ge hwolssin deo joh-eun geo gat-a.
FastPitch (Baseline) EGW + DW (Ours) EGW + DW + HPC (Ours)
 
Sentence: 한참 연주 결혼식 준비할 때도 니가 그때 얘가 나쁜 애는 아닌데 지금 상황이 너무 힘들어서 그렇다고 그랬잖아.
(Pronunciation): Hancham yeonju gyeolhonsig junbihal ttaedo niga geuttae yaega nappeun aeneun aninde jigeum sanghwang-i neomu himdeul-eoseo geuleohdago geulaessjanh-a.
FastPitch (Baseline) EGW + DW (Ours) EGW + DW + HPC (Ours)
 
Sentence: 예를 들어 상추쌈을 같이 해서 먹는 다든가 하여튼 채소를 많이 먹으라 그랬던거지.
(Pronunciation): Yeleul deul-eo sangchussam-eul gat-i haeseo meogneun dadeunga hayeoteun chaesoleul manh-i meog-eula geulaessdeongeoji.
FastPitch (Baseline) EGW + DW (Ours) EGW + DW + HPC (Ours)
 
Sentence: 일단 도착하자마자 숙소에 짐 풀고 우리 둘은 샤워하고 애들이랑 바다 가서 술 마시고 놀았지.
(Pronunciation): Ildan dochaghajamaja sugso-e jim pulgo uli dul-eun syawohago aedeul-ilang bada gaseo sul masigo nol-assji.
FastPitch (Baseline) EGW + DW (Ours) EGW + DW + HPC (Ours)