-
An Analysis of How Many Undiscovered Vulnerabilities Remain in Information Systems
Authors:
Jonathan M Spring
Abstract:
Vulnerability management strategy, from both organizational and public policy perspectives, hinges on an understanding of the supply of undiscovered vulnerabilities. If the number of undiscovered vulnerabilities is small enough, then a reasonable investment strategy would be to focus on finding and removing the remaining undiscovered vulnerabilities. If the number of undiscovered vulnerabilities i…
▽ More
Vulnerability management strategy, from both organizational and public policy perspectives, hinges on an understanding of the supply of undiscovered vulnerabilities. If the number of undiscovered vulnerabilities is small enough, then a reasonable investment strategy would be to focus on finding and removing the remaining undiscovered vulnerabilities. If the number of undiscovered vulnerabilities is and will continue to be large, then a better investment strategy would be to focus on quick patch dissemination and engineering resilient systems. This paper examines a paradigm, namely that the number of undiscovered vulnerabilities is manageably small, through the lens of mathematical concepts from the theory of computing. From this perspective, we find little support for the paradigm of limited undiscovered vulnerabilities. We then briefly support the notion that these theory-based conclusions are relevant to practical computers in use today. We find no reason to believe undiscovered vulnerabilities are not essentially unlimited in practice and we examine the possible economic impacts should this be the case. Based on our analysis, we recommend vulnerability management strategy adopts an approach favoring quick patch dissemination and engineering resilient systems, while continuing good software engineering practices to reduce (but never eliminate) vulnerabilities in information systems.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
On managing vulnerabilities in AI/ML systems
Authors:
Jonathan M. Spring,
April Galyardt,
Allen D. Householder,
Nathan VanHoudnos
Abstract:
This paper explores how the current paradigm of vulnerability management might adapt to include machine learning systems through a thought experiment: what if flaws in machine learning (ML) were assigned Common Vulnerabilities and Exposures (CVE) identifiers (CVE-IDs)? We consider both ML algorithms and model objects. The hypothetical scenario is structured around exploring the changes to the six…
▽ More
This paper explores how the current paradigm of vulnerability management might adapt to include machine learning systems through a thought experiment: what if flaws in machine learning (ML) were assigned Common Vulnerabilities and Exposures (CVE) identifiers (CVE-IDs)? We consider both ML algorithms and model objects. The hypothetical scenario is structured around exploring the changes to the six areas of vulnerability management: discovery, report intake, analysis, coordination, disclosure, and response. While algorithm flaws are well-known in the academic research community, there is no apparent clear line of communication between this research community and the operational communities that deploy and manage systems that use ML. The thought experiments identify some ways in which CVE-IDs may establish some useful lines of communication between these two communities. In particular, it would start to introduce the research community to operational security concepts, which appears to be a gap left by existing efforts.
△ Less
Submitted 22 January, 2021;
originally announced January 2021.
-
Learning the Associations of MITRE ATT&CK Adversarial Techniques
Authors:
Rawan Al-Shaer,
Jonathan M. Spring,
Eliana Christou
Abstract:
The MITRE ATT&CK Framework provides a rich and actionable repository of adversarial tactics, techniques, and procedures (TTP). However, this information would be highly useful for attack diagnosis (i.e., forensics) and mitigation (i.e., intrusion response) if we can reliably construct technique associations that will enable predicting unobserved attack techniques based on observed ones. In this pa…
▽ More
The MITRE ATT&CK Framework provides a rich and actionable repository of adversarial tactics, techniques, and procedures (TTP). However, this information would be highly useful for attack diagnosis (i.e., forensics) and mitigation (i.e., intrusion response) if we can reliably construct technique associations that will enable predicting unobserved attack techniques based on observed ones. In this paper, we present our statistical machine learning analysis on APT and Software attack data reported by MITRE ATT&CK to infer the technique clustering that represents the significant correlation that can be used for technique prediction. Due to the complex multidimensional relationships between techniques, many of the traditional clustering methods could not obtain usable associations. Our approach, using hierarchical clustering for inferring attack technique associations with 95% confidence, provides statistically significant and explainable technique correlations. Our analysis discovers 98 different technique associations (i.e., clusters) for both APT and Software attacks. Our evaluation results show that 78% of the techniques associated by our algorithm exhibit significant mutual information that indicates reasonably high predictability.
△ Less
Submitted 12 May, 2020; v1 submitted 16 April, 2020;
originally announced May 2020.
-
Review of human decision-making during computer security incident analysis
Authors:
Jonathan M. Spring,
Phyllis Illari
Abstract:
We review practical advice on decision-making during computer security incident response. Scope includes standards from the IETF, ISO, FIRST, and the US intelligence community. To focus on human decision-making, the scope is the evidence collection, analysis, and reporting phases of response. The results indicate both strengths and gaps. A strength is available advice on how to accomplish many spe…
▽ More
We review practical advice on decision-making during computer security incident response. Scope includes standards from the IETF, ISO, FIRST, and the US intelligence community. To focus on human decision-making, the scope is the evidence collection, analysis, and reporting phases of response. The results indicate both strengths and gaps. A strength is available advice on how to accomplish many specific tasks. However, there is little guidance on how to prioritize tasks in limited time or how to interpret, generalize, and convincingly report results. Future work should focus on these gaps in explication and specification of decision-making during incident analysis.
△ Less
Submitted 24 March, 2019;
originally announced March 2019.
