Mel-FullSubNet
Abstract
Rui Zhou, Xian Li, Ying Fang and Xiaofei Li. Mel-FullSubNet: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR. [Code], [Pdf]
In this work, we propose Mel-FullSubNet, a single-channel Mel-spectrogram denoising and dereverberation network for improving both speech quality and automatic speech recognition (ASR) performance. Mel-FullSubNet takes as input the noisy and reverberant Mel-spectrogram and predicts the corresponding clean Mel-spectrogram. The enhanced Mel-spectrogram can be either transformed to speech waveform with a neural vocoder or directly used for ASR. Mel-FullSubNet encapsulates interleaved full-band and sub-band networks, for learning the full-band spectral pattern of signals and the sub-band/narrow-band properties of signals, respectively. Compared to linear-frequency domain or time-domain speech enhancement, the major advantage of Mel-spectrogram enhancement is that Mel-frequency presents speech in a more compact way and thus is easier to learn, which will benefit both speech quality and ASR. Experimental results demonstrate a significant improvement in both speech quality and ASR performance achieved by the proposed model.
Examples
Please open this page with Edge or Chrome, and not use Firefox. The audio playing is problematic in Firefox.
| Setting | Model | REVERB | DNS | CHIME-4 | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unpro. | ||||||||||||||||||
| Target | ||||||||||||||||||
| Online | FullSubNet | |||||||||||||||||
| Mel-FullSubNet | ||||||||||||||||||
| Offline | FullSubNet | |||||||||||||||||
| VoiceFixer | ||||||||||||||||||
| DMSEbase | ||||||||||||||||||
| Mel-FullSubNet | ||||||||||||||||||