Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag Challenges and Tabular Q-Learning

Published on May 26, 2020in arXiv: Cryptography and Security
Fabio Massimo Zennaro2
Estimated H-index: 2
(University of Oslo),
Laszlo Erdodi1
Estimated H-index: 1
(University of Oslo)
Penetration testing is a security exercise aimed at assessing the security of a system by simulating attacks against it. So far, penetration testing has been carried out mainly by trained human attackers and its success critically depended on the available expertise. Automating this practice constitutes a non-trivial problem, as the range of actions that a human expert may attempts against a system and the range of knowledge she relies on to take her decisions are hard to capture. In this paper, we focus our attention on simplified penetration testing problems expressed in the form of capture the flag hacking challenges, and we apply reinforcement learning algorithms to try to solve them. In modelling these capture the flag competitions as reinforcement learning problems we highlight the specific challenges that characterize penetration testing. We observe these challenges experimentally across a set of varied simulations, and we study how different reinforcement learning techniques may help us addressing these challenges. In this way we show the feasibility of tackling penetration testing using reinforcement learning, and we highlight the challenges that must be taken into consideration, and possible directions to solve them.
  • References (17)
  • Citations (1)
📖 Papers frequently viewed together
2017SAFECOMP: International Conference on Computer Safety, Reliability, and Security
40 Citations
2011ICSECS: International Conference on Software Engineering and Computer Systems
3 Citations
78% of Scinapse members use related papers. After signing in, all features are FREE.
#1John A. Bland (UAH: University of Alabama in Huntsville)H-Index: 2
#2Mikel D. Petty (UAH: University of Alabama in Huntsville)H-Index: 1
Last. Walter Alan Cantrell (Lipscomb University)H-Index: 2
view all 5 authors...
Abstract Cybersecurity is an increasingly important challenge for computer systems. In this work, cyberattacks were modeled using an extension of the well-known Petri net formalism. That formalism, designated Petri nets with players, strategies, and costs, models the states of the cyberattack and events during the attack as markings and transition firings in the net respectively. The formalism models the attacker and defender as competing players who may observe the marking of a subset of the ne...
2 CitationsSource
#2Eduardo AlonsoH-Index: 1
Last. Kevin D. JonesH-Index: 8
view all 5 authors...
A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns...
1 CitationsSource
#1Adrià Puigdomènech Badia (Google)H-Index: 8
#2Bilal PiotH-Index: 17
Last. Charles BlundellH-Index: 26
view all 7 authors...
Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing outstandingly well on many games of the set, but very poorly in several of the most challenging games. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games. To achieve this result, we train a n...
9 Citations
Penetration testing (also known as pentesting or PT) is a common practice for actively assessing the defenses of a computer network by planning and executing all possible attacks to discover and exploit existing vulnerabilities. Current penetration testing methods are increasingly becoming non-standard, composite and resource-consuming despite the use of evolving tools. In this paper, we propose and evaluate an AI-based pentesting system which makes use of machine learning techniques, namely rei...
2 CitationsSource
Apr 8, 2019 in SAC (ACM Symposium on Applied Computing)
#1Patrick Speicher (Saarland University)H-Index: 2
#2Marcel Steinmetz (Saarland University)H-Index: 6
Last. Robert Kunnemann (Saarland University)H-Index: 2
view all 5 authors...
Penetration testing is a well-established practical concept for the identification of potentially exploitable security weaknesses and an important component of a security audit. Providing a holistic security assessment for networks consisting of several hundreds hosts is hardly feasible though without some sort of mechanization. Mitigation, prioritizing counter-measures subject to a given budget, currently lacks a solid theoretical understanding and is hence more art than science. In this work, ...
2 CitationsSource
#1Peter W. BattagliaH-Index: 28
#2Jessica B. HamrickH-Index: 13
Last. Razvan PascanuH-Index: 51
view all 27 authors...
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hall...
508 Citations
The study and understanding of human behaviour is relevant to computer science, artificial intelligence, neural computation, cognitive science, philosophy, psychology, and several other areas. Presupposing cognition as basis of behaviour, among the most prominent tools in the modelling of behaviour are computational-logic systems, connectionist models of cognition, and models of uncertainty. Recent studies in cognitive science, artificial intelligence, and psychology have produced a number of co...
39 Citations
#1David SilverH-Index: 58
Last. Demis HassabisH-Index: 53
view all 17 authors...
Starting from zero knowledge and without human data, AlphaGo Zero was able to teach itself to play Go and to develop novel strategies that provide new insights into the oldest of games.
2,251 CitationsSource
Dec 5, 2016 in ACSAC (Annual Computer Security Applications Conference)
#1Andy Applebaum (MITRE: Mitre Corporation)H-Index: 5
#2Doug Miller (MITRE: Mitre Corporation)H-Index: 1
Last. Ross D. Wolf (MITRE: Mitre Corporation)H-Index: 1
view all 5 authors...
Red teams play a critical part in assessing the security of a network by actively probing it for weakness and vulnerabilities. Unlike penetration testing - which is typically focused on exploiting vulnerabilities - red teams assess the entire state of a network by emulating real adversaries, including their techniques, tactics, procedures, and goals. Unfortunately, deploying red teams is prohibitive: cost, repeatability, and expertise all make it difficult to consistently employ red team tests. ...
11 CitationsSource
#1Greg BrockmanH-Index: 2
#2Vicki CheungH-Index: 4
Last. Wojciech ZarembaH-Index: 33
view all 7 authors...
925 Citations
Cited By1