Performance comparison of random forests and extremely randomized trees

Abstract

Random Forests (RF) recently have gained significant attention in the scientific community as simple, versatile and efficient machine learning algorithm. It has been used for variety of tasks due it its high predictive performance, ability to perform feature ranking, its simple parallelization, and due to its low sensitivity to parameter tuning. In recent years another treebased ensemble method has been proposed, namely the Extremely Randomized Trees (ERT). These trees by definition have similar properties. However, there is no extensive empirical evaluation of both algorithms that would identify strengths and weaknesses of each of them. In this paper we evaluate both algorithms of several publicly available datasets. Our experiments show that ERT are faster as the dataset size increases and can provide at least the same level of predictive performance. As for feature ranking capabilities, we have statistically confirmed that both provide the same ranking, provided that the number of trees is large enough.