The recent success of applying state-of-the-art AI algorithms on tasks with modern big data has raised concerns on their efficiency. For instance, ensemble methods like random forest usually require lots of sub-learners to achieve favorable predictive performance, which result in an increasing demand for model storage space with an increasing dataset size.
Such tree-based models may go to GB level, making them difficult to deploy on resource-constrained devices or costly in the cloud. Moreover, a larger model usually causes a higher inference time and energy consumption which are not usable in many real-world applications.