Visually-derived Wiener filters for speech enhancement

Ibrahim Almajai; Ben P. Milner; Jonathan Darch; Saeed V. Vaseghi

doi:10.1109/ICASSP.2007.366980

Visually-derived Wiener filters for speech enhancement

Ibrahim Almajai, Ben P. Milner, Jonathan Darch, Saeed V. Vaseghi

Research output: Contribution to conference › Paper

15 Citations (Scopus)

Abstract

This work begins by examining the correlation between audio and visual speech features and reveals higher correlation to exist within individual phoneme sounds rather than globally across all speech. Utilising this correlation, a visually-derived Wiener filter is proposed in which clean power spectrum estimates are obtained from visual speech features. Two methods of extracting clean power spectrum estimates are made; first from a global estimate using a single Gaussian mixture model (GMM), and second from phoneme-specific estimates using a hidden Markov model (HMM)-GMM structure. Measurement of estimation accuracy reveals that the phoneme-specific (HMM-GMM) system leads to lower estimation errors than the global (GMM) system. Finally, the effectiveness of visually-derived Wiener filtering is examined

Original language	English
Pages	IV-585-IV-588
Number of pages	4
DOIs	https://doi.org/10.1109/ICASSP.2007.366980
Publication status	Published - 2007
Event	IEEE International Conference on Acoustics, Speech and Signal Processing - Honolulu, United States Duration: 15 Apr 2007 → 20 Apr 2007

Conference

Conference	IEEE International Conference on Acoustics, Speech and Signal Processing
Country/Territory	United States
City	Honolulu
Period	15/04/07 → 20/04/07

Access to Document

10.1109/ICASSP.2007.366980

Cite this

@conference{9e96fa2dafd74004ac3d57cb6968d616,

title = "Visually-derived Wiener filters for speech enhancement",

abstract = "This work begins by examining the correlation between audio and visual speech features and reveals higher correlation to exist within individual phoneme sounds rather than globally across all speech. Utilising this correlation, a visually-derived Wiener filter is proposed in which clean power spectrum estimates are obtained from visual speech features. Two methods of extracting clean power spectrum estimates are made; first from a global estimate using a single Gaussian mixture model (GMM), and second from phoneme-specific estimates using a hidden Markov model (HMM)-GMM structure. Measurement of estimation accuracy reveals that the phoneme-specific (HMM-GMM) system leads to lower estimation errors than the global (GMM) system. Finally, the effectiveness of visually-derived Wiener filtering is examined",

author = "Ibrahim Almajai and Milner, {Ben P.} and Jonathan Darch and Vaseghi, {Saeed V.}",

year = "2007",

doi = "10.1109/ICASSP.2007.366980",

language = "English",

pages = "IV--585--IV--588",

note = "IEEE International Conference on Acoustics, Speech and Signal Processing ; Conference date: 15-04-2007 Through 20-04-2007",

}

TY - CONF

T1 - Visually-derived Wiener filters for speech enhancement

AU - Almajai, Ibrahim

AU - Milner, Ben P.

AU - Darch, Jonathan

AU - Vaseghi, Saeed V.

PY - 2007

Y1 - 2007

N2 - This work begins by examining the correlation between audio and visual speech features and reveals higher correlation to exist within individual phoneme sounds rather than globally across all speech. Utilising this correlation, a visually-derived Wiener filter is proposed in which clean power spectrum estimates are obtained from visual speech features. Two methods of extracting clean power spectrum estimates are made; first from a global estimate using a single Gaussian mixture model (GMM), and second from phoneme-specific estimates using a hidden Markov model (HMM)-GMM structure. Measurement of estimation accuracy reveals that the phoneme-specific (HMM-GMM) system leads to lower estimation errors than the global (GMM) system. Finally, the effectiveness of visually-derived Wiener filtering is examined

AB - This work begins by examining the correlation between audio and visual speech features and reveals higher correlation to exist within individual phoneme sounds rather than globally across all speech. Utilising this correlation, a visually-derived Wiener filter is proposed in which clean power spectrum estimates are obtained from visual speech features. Two methods of extracting clean power spectrum estimates are made; first from a global estimate using a single Gaussian mixture model (GMM), and second from phoneme-specific estimates using a hidden Markov model (HMM)-GMM structure. Measurement of estimation accuracy reveals that the phoneme-specific (HMM-GMM) system leads to lower estimation errors than the global (GMM) system. Finally, the effectiveness of visually-derived Wiener filtering is examined

U2 - 10.1109/ICASSP.2007.366980

DO - 10.1109/ICASSP.2007.366980

M3 - Paper

SP - IV-585-IV-588

T2 - IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 15 April 2007 through 20 April 2007

ER -