Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

Published on Apr 16, 2020
K. Pozdniakov1
Estimated H-index: 1
E. Alonso1
Estimated H-index: 1
+ 2 AuthorsKevin D. Jones7
Estimated H-index: 7
A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach.
  • References (0)
  • Citations (1)
📖 Papers frequently viewed together
5 Citations
2 Citations
2018FIT: Frontiers of Information Technology
4 Authors (Faiq Khalid, ..., Muhammad Shafique)
1 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
Cited By1
#1Fabio Massimo Zennaro (University of Oslo)H-Index: 2
#2Laszlo Erdodi (University of Oslo)
Penetration testing is a security exercise aimed at assessing the security of a system by simulating attacks against it. So far, penetration testing has been carried out mainly by trained human attackers and its success critically depended on the available expertise. Automating this practice constitutes a non-trivial problem, as the range of actions that a human expert may attempts against a system and the range of knowledge she relies on to take her decisions are hard to capture. In this paper,...