Implementing a Hybrid Model using K-Means Clustering and Artificial Neural Networks for Risk Prediction in Life Insurance

Abstract

Accurate assessment of policy holder risk is critical for life insurance companies to properly price policies and manage long-term liabilities. However, the complexity of risk factors makes reliance solely on traditional actuarial models inadequate, especially with the proliferation of big data and unstandardized data from diverse sources. This study investigated the development and performance of a hybrid machine learning model combining artificial neural networks (ANN) and K-means clustering for enhanced risk prediction in life insurance underwriting. The exponential growth of unlabelled data presented challenges for predictive modelling. The proposed hybrid model leveraged the strengths of artificial neural networks in modelling nonlinear relationships and K-means clustering in unsupervised for pattern recognition to handle unstandardized data. Using anonymized life insurance application data from Kaggle, the hybrid model was evaluated against the artificial neural network algorithm alone. The results demonstrated that integrating K-means clustering and artificial neural network together with principal component analysis for pre-processing led to superior model performance, with testing accuracy improving from 90% for artificial neural network to 98% for the hybrid technique. Additional metrics like precision, recall and AUC also showed enhancements. The improved predictive capability highlighted the potential of the hybrid approach in transforming legacy underwriting practices towards a more sophisticated data-driven analytical evaluation of policy holder risk. However, limitations existed including the use of single sourced insurance dataset due to data privacy concerns. Further research on integrating diverse algorithms can help insurers unlock more value and gain a competitive edge through advanced analytical modelling and testing on larger real-world datasets. While challenges remain, this study provided key insights into a promising new technique for modernizing risk prediction in the life insurance industry in the era of big data.