TY - GEN
T1 - Incremental hacker forum exploit collection and classification for proactive cyber threat intelligence
T2 - 16th IEEE International Conference on Intelligence and Security Informatics, ISI 2018
AU - Williams, Ryan
AU - Samtani, Sagar
AU - Patton, Mark
AU - Chen, Hsinchun
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/12/24
Y1 - 2018/12/24
N2 - Cyber threats have emerged as a key societal concern. To counter the growing threat of cyber-attacks, organizations, in recent years, have begun investing heavily in developing Cyber Threat Intelligence (CTI). Fundamentally a data driven process, many organizations have traditionally collected and analyzed data from internal log files, resulting in reactive CTI. The online hacker community can offer significant proactive CTI value by alerting organizations to threats they were not previously aware of. Amongst various platforms, forums provide the richest metadata, data permanence, and tens of thousands of freely available Tools, Techniques, and Procedures (TTP). However, forums often employ anti-crawling measures such as authentication, throttling, and obfuscation. Such limitations have restricted many researchers to batch collections. This exploratory study aims to (1) design a novel web crawler augmented with numerous anti-crawling countermeasures to collect hacker exploits on an ongoing basis, (2) employ a state-of-the-art deep learning approach, Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN), to automatically classify exploits into pre-defined categories on-the-fly, and (3) develop interactive visualizations enabling CTI practitioners and researchers to explore collected exploits for proactive, timely CTI. The results of this study indicate, among other findings, that system and network exploits are shared significantly more than other exploit types.
AB - Cyber threats have emerged as a key societal concern. To counter the growing threat of cyber-attacks, organizations, in recent years, have begun investing heavily in developing Cyber Threat Intelligence (CTI). Fundamentally a data driven process, many organizations have traditionally collected and analyzed data from internal log files, resulting in reactive CTI. The online hacker community can offer significant proactive CTI value by alerting organizations to threats they were not previously aware of. Amongst various platforms, forums provide the richest metadata, data permanence, and tens of thousands of freely available Tools, Techniques, and Procedures (TTP). However, forums often employ anti-crawling measures such as authentication, throttling, and obfuscation. Such limitations have restricted many researchers to batch collections. This exploratory study aims to (1) design a novel web crawler augmented with numerous anti-crawling countermeasures to collect hacker exploits on an ongoing basis, (2) employ a state-of-the-art deep learning approach, Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN), to automatically classify exploits into pre-defined categories on-the-fly, and (3) develop interactive visualizations enabling CTI practitioners and researchers to explore collected exploits for proactive, timely CTI. The results of this study indicate, among other findings, that system and network exploits are shared significantly more than other exploit types.
KW - CTI
KW - Cyber threat intelligence
KW - Hacker exploits
KW - Hacker forum
KW - Recurrent neural network
KW - Web crawling
UR - http://www.scopus.com/inward/record.url?scp=85061054594&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85061054594&partnerID=8YFLogxK
U2 - 10.1109/ISI.2018.8587336
DO - 10.1109/ISI.2018.8587336
M3 - Conference contribution
AN - SCOPUS:85061054594
T3 - 2018 IEEE International Conference on Intelligence and Security Informatics, ISI 2018
SP - 94
EP - 99
BT - 2018 IEEE International Conference on Intelligence and Security Informatics, ISI 2018
A2 - Lee, Dongwon
A2 - Mezzour, Ghita
A2 - Kumaraguru, Ponnurangam
A2 - Saxena, Nitesh
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 9 November 2018 through 11 November 2018
ER -