Lip-reading enhancement for law enforcement

Barry Theobald, Richard Harvey, Stephen Cox, Colin Lewis, Gari Owens

Research output: Contribution to conferencePaper

8 Citations (Scopus)


Accurate lip-reading techniques would be of enormous benefit for agencies involved in counter-terrorism and other law-enforcement areas. Unfortunately, there are very few skilled lip-readers, and it is apparently a difficult skill to transmit, so the area is under-resourced. In this paper we investigate the possibility of making the lip-reading task more amenable to a wider range of operators by enhancing lip movements in video sequences using active appearance models. These are generative, parametric models commonly used to track faces in images and video sequences. The parametric nature of the model allows a face in an image to be encoded in terms of a few tens of parameters, while the generative nature allows faces to be re-synthesised using the parameters. The aim of this study is to determine if exaggerating lip-motions in video sequences by amplifying the parameters of the model improves lip-reading ability. We also present results of lip-reading tests undertaken by experienced (but non-expert) adult subjects who claim to use lip-reading in their speech recognition process. The results, which are comparisons of word error-rates on unprocessed and processed video, are mixed. We find that there appears to be the potential to improve the word error rate but, for the method to improve the intelligibility there is need for more sophisticated tracking and visual modelling. Our technique can also act as an expression or visual gesture amplifier and so has applications to animation and the presentation of information via avatars or synthetic humans
Original languageEnglish
Number of pages5
Publication statusPublished - Sep 2006
EventProceedings of SPIE 6402 Conference on Optics and Photonics for Counterterrorism and Crime Fighting - Stockholm, Sweden
Duration: 11 Sep 2006 → …


ConferenceProceedings of SPIE 6402 Conference on Optics and Photonics for Counterterrorism and Crime Fighting
Period11/09/06 → …

Cite this