Investigations on the Characteristics of Random Decision Tree Ensembles

An ensemble is viewed as a machine learning system that combines multiple models to work collectively in the hope of producing a better performance than that of individuals. However, an ensemble's accuracy cannot be easily determined as it involves several factors, e.g. individual model's accuracy, diversity between its member models, decision- making strategy and number of members and the relationships between them are unclear. This paper, taking random decision tree ensembles as testing platforms, investigates these relationships and the strategies for creating ensembles from randomly generated trees. Specifically, we devised three sets of procedures for conducting experiments using twelve data sets from the UCI repository to determine the importance of individual model accuracy and the diversity between decision tree models within an ensemble. The main findings of the investigations are presented and discussed in the paper.