Linear regression motion analysis for unsupervised temporal segmentation of human actions

One of the biggest dificulties in human action analysis is the temporal complexity and structure of actions. By breaking actions down into smaller temporal pieces, it may be possible to enhance action recognition, or allow unsupervised temporal action clustering. We propose a temporal segmentation system for human action recognition based on person tracking and a novel segmentation algorithm. We apply optical flow, PCA, and linear regression error estimation to human action videos to get a metric, L′, that can be used to split an action into several more easily recognised subactions. The L′ metric can be effectively calculated and is robust. To validate the semantic coherence of the sub-actions, we represent the sub-actions as features using a variant of the Motion History Image and perform action recognition experiments on two popular datasets, the KTH and the MSR2. Our results demonstrate that the algorithm performs well, showing promise for future application in action clustering and action retrieval tasks.