HMBO-LDC: A Hybrid Model Employing Reinforcement Learning with Bayesian Optimization for Long Document Classification

Abstract

With the emergence of distributed computing platforms and cloud-big data eco-system, there has been increased growth of textual documents stored in cloud infrastructure. It is observed that most of the documents happened to be lengthy. Automatic classification of such documents is made possible with deep learning models. However, it is observed that deep learning models like CNN and its variants do have many hyper parameters that are to be optimized in order to leverage classification performance. The existing optimization methods based on random search are found to have suboptimal performance when compared with Bayesian Optimization (BO). However, BO has issues pertaining to choice of covariance function, time consumption and support for multi-core parallelism. To address these limitations, we proposed an algorithm named Enhanced Bayesian Optimization (EBO) designed to optimize hyper parameter tuning. We also proposed another algorithm known as Hybrid Model with Bayesian Optimization for Long Document Classification (HMBO-LDC). The latter invokes the former appropriately in order to improve parameter optimization of the proposed hybrid model prior to performing long document classification. HMBO-LDC is evaluated and compared against existing models such as CNN feature aggregation method, CNN with LSTM and CNN with recurrent attention model. Experimental results revealed that HMBO-LDC outperforms other methods with highest classification accuracy 98.76%.