Snowball-Miner: Integration of Deep Learning for Extraction of Cyber Threat Intelligence from Dark Web

Abstract

In Cyber threat intelligence is a crucial component in defending against cybersecurity threats. Cyber security dark web, security Blogs, Hackers’ community, news forums, Open-Source Intelligence (OSINT) are known as the harbor of illicit activities and serve as a breeding ground for cybercriminals. Extracting actionable intelligence from the dark web is challenging due to its anonymous and encrypted nature. State-of-art work proposed machine learning and deep learning approach to aggregate the dark web for cyber threat intelligence from data present in the dark web.  This paper proposes, a novel approach utilizing Snowball-Miner for cyber threat intelligence discovery from the dark web. The model is trained on a diverse dataset consisting of dark web forums, hidden .onion based marketplaces and other underground platforms using Snowball-crawler. However, we have employed hybrid convolutional model CNN-LSTM and CNN-GRU adopting doc2vec word embedding to classify into four domains viz Energy Sector, Finance, Illicit Activities and illegal Services. From our experiment it emerged that, CNN-LSTM outperforms as 96.37% for classification of domain specific threat documents. Furthermore, after data preparation we implemented NLP technique and extracted the domain specific Indicator of Compromise (IoCs) using RegEx parser and Subject, Object and Verb (SOV) semantics dependency analysis. Finally, we have integrated IoCs and Threat keywords with respective domains to generate domain specific threat intelligence which enhance the quality of the domain specific CTI based on R-dimension (Relevance).