Inferring the structure of a tennis game using audio information

We describe a novel framework for inferring the low-level structure of a sports game (tennis) using only the information available on the audio track of a video recording of the game. Our goal is to segment the games into a sequence of points, the natural unit for describing a tennis match. The framework is hierarchical, consisting of, at the lowest level, identification of audio events, followed by match (i.e. semantic) events, and at the highest level, game points. Different techniques that are appropriate to the characteristics of each of these events are used to detect them, and these techniques are coupled in a probabilistic framework. The techniques consist of Gaussian mixture models and a hierarchical language model to detect sequences of audio events, a maximum entropy Markov model to infer match events from these audio events, and multigrams to infer the segmentation of a sequence of match events into sequences of points in a a tennis game. Our results are promising, giving an F-score for the final detection of points of > 0:7.