PeriodGrad: Towards Pitch-Controllable Neural Vocoder Based on a Diffusion Probabilistic Model
Yukiya Hono, Kei Hashimoto, Yoshihiko Nankaku, and Keiichi Tokuda
Nagoya Institute of Technology, Nagoya, Japan
Accepted for ICASSP 2024
Abstruct
This paper presents a neural vocoder based on a denoising diffusion probabilistic model (DDPM) incorporating explicit periodic signals as auxiliary conditioning signals. Recently, DDPM-based neural vocoders have gained prominence as non-autoregressive models that can generate high-quality waveforms. The neural vocoders based on DDPM have the advantage of training with a simple time-domain loss. In practical applications, such as singing voice synthesis, there is a demand for neural vocoders to generate high-fidelity speech waveforms with flexible pitch control. However, conventional DDPM-based neural vocoders struggle to generate speech waveforms under such conditions. Our proposed model aims to accurately capture the periodic structure of speech waveforms by incorporating explicit periodic signals. Experimental results show that our model improves sound quality and provides better pitch control than conventional DDPM-based neural vocoders.
Experimental results
Methods
We compared three models. Each model was trained on two auxiliary feature sets.
Model details:- PeriodNet: A pitch-controllable GAN-based neural vocoder with a parallel structure [1].
- PriorGrad: A DDPM-based baseline neural vocoder [2].
- PeriodGrad: The proposed DDPM-based neural vocoder.
- voc: WORLD vocoder parameter set (mel-cepstral coefficient, continuous log F0, aperiodicity, and V/UV binary flag).
- ms+F0: Log mel-spectrograms, continuous log F0, and V/UV binary flag.
Objective evaluation results
Subjective evaluation results
Results of MOS test with 95% confidence intervals. The methods annotated with * have insufficient pitch control performance. These methods are impractical, even if the subjective rating of sound quality could be better.
Audio samples (Japanese)
$F_0$ condition: -12 [semitones] (=$0.5 \times F_0$)
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: -9 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: -6 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: -3 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: +0 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| Natural | ||
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: +3 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: +6 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: +9 [semitones]
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
$F_0$ condition: +12 [semitones] (=$2.0 \times F_0$)
| System | Sample 1 | Sample 2 |
|---|---|---|
| PeriodNet (voc) | ||
| PriorGrad (voc) | ||
| PeriodGrad (voc) | ||
| PeriodNet (ms+F0) | ||
| PriorGrad (ms+F0) | ||
| PeriodGrad (ms+F0) |
References
[1] Y. Hono, S. Takaki, K. Hashimoto, K. Oura, Y. Nankaku, and K. Tokuda, “PeriodNet: A non-autoregressive raw waveform generative model with a structure separating periodic and aperiodic components,” IEEE Access, vol. 9, pp. 137599-137612, 2021.
[2] S.-g. Lee, H. Kim, C. Shin, X. Tan, C. Liu, Q. Meng, T. Qin, W. Chen, S. Yoon, and T.-Y. Liu, “PriorGrad: Improving conditional denoising diffusion models with data-dependent adaptive prior,” in Proc. ICLR, 2022.