Speech and Language Processing for a Constrained Speech Translation System

A system that provides translation from speech to sign language (TESSA) is described. TESSA has been developed to assist a Post Office clerk (who has no knowledge of sign language) in making a transaction with a customer using sign language. The system uses a set of about 370 pre-defined phrases which have been pre-stored and can be signed by a specially-developed avatar. The clerk is unable to memorise all these phrases and so the system attempts to map his or her input speech to the pre-stored phrase that is semantically equivalent to the input phrase. We describe the language processing that was developed to perform this task, and give results obtained using alternative formulations of the phrases from a number of speakers. We then give results for speech input and show the effect of different configurations of the recogniser's language model on the results. Best performance was obtained when the language model was closely integrated with the task.