Hybrid Ensembles and Coincident-Failure Diversity

This paper presents an approach for constructing hybrid ensembles with two categories of members: trained neural networks and decision trees in the hope of increasing diversity and reducing coincident failures. The diversity among the members of an ensemble has been generally recognised as a key factor for improving the overall performance of the ensemble. Of the two heterogeneous members evaluated with a number of the diversity measures, we found that there is a statistically low level coincident-failure diversity among homogeneous members but a relatively high level diversity between heterogeneous members. This provides theoretical basis for constructing hybrid ensembles with members developed by different methodologies. The hybrid ensembles have been built for a real-world problem: predicting training injury of military recruits. Their performances are evaluated in terms of diversity, reliability and prediction accuracy, and also compared with the homogeneous ensembles of neural nets or decision trees. The results indicate that the hybrid ensembles have a considerably high level diversity and thus are able to produce a better performance