-
Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks
Authors:
Mohamad Fazelnia,
Viktoria Koscinski,
Spencer Herzog,
Mehdi Mirakhorli
Abstract:
We investigate the use of Natural Language Inference (NLI) in automating requirements engineering tasks. In particular, we focus on three tasks: requirements classification, identification of requirements specification defects, and detection of conflicts in stakeholders' requirements. While previous research has demonstrated significant benefit in using NLI as a universal method for a broad spectr…
▽ More
We investigate the use of Natural Language Inference (NLI) in automating requirements engineering tasks. In particular, we focus on three tasks: requirements classification, identification of requirements specification defects, and detection of conflicts in stakeholders' requirements. While previous research has demonstrated significant benefit in using NLI as a universal method for a broad spectrum of natural language processing tasks, these advantages have not been investigated within the context of software requirements engineering. Therefore, we design experiments to evaluate the use of NLI in requirements analysis. We compare the performance of NLI with a spectrum of approaches, including prompt-based models, conventional transfer learning, Large Language Models (LLMs)-powered chatbot models, and probabilistic models. Through experiments conducted under various learning settings including conventional learning and zero-shot, we demonstrate conclusively that our NLI method surpasses classical NLP methods as well as other LLMs-based and chatbot models in the analysis of requirements specifications. Additionally, we share lessons learned characterizing the learning settings that make NLI a suitable approach for automating requirements engineering tasks.
△ Less
Submitted 24 April, 2024;
originally announced May 2024.
-
A Landscape Study of Open Source and Proprietary Tools for Software Bill of Materials (SBOM)
Authors:
Mehdi Mirakhorli,
Derek Garcia,
Schuyler Dillon,
Kevin Laporte,
Matthew Morrison,
Henry Lu,
Viktoria Koscinski,
Christopher Enoch
Abstract:
Modern software applications heavily rely on diverse third-party components, libraries, and frameworks sourced from various vendors and open source repositories, presenting a complex challenge for securing the software supply chain. To address this complexity, the adoption of a Software Bill of Materials (SBOM) has emerged as a promising solution, offering a centralized repository that inventories…
▽ More
Modern software applications heavily rely on diverse third-party components, libraries, and frameworks sourced from various vendors and open source repositories, presenting a complex challenge for securing the software supply chain. To address this complexity, the adoption of a Software Bill of Materials (SBOM) has emerged as a promising solution, offering a centralized repository that inventories all third-party components and dependencies used in an application. Recent supply chain breaches, exemplified by the SolarWinds attack, underscore the urgent need to enhance software security and mitigate vulnerability risks, with SBOMs playing a pivotal role in this endeavor by revealing potential vulnerabilities, outdated components, and unsupported elements. This research paper conducts an extensive empirical analysis to assess the current landscape of open-source and proprietary tools related to SBOM. We investigate emerging use cases in software supply chain security and identify gaps in SBOM technologies. Our analysis encompasses 84 tools, providing a snapshot of the current market and highlighting areas for improvement.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
A Novel Approach to Identify Security Controls in Source Code
Authors:
Ahmet Okutan,
Ali Shokri,
Viktoria Koscinski,
Mohamad Fazelinia,
Mehdi Mirakhorli
Abstract:
Secure by Design has become the mainstream development approach ensuring that software systems are not vulnerable to cyberattacks. Architectural security controls need to be carefully monitored over the software development life cycle to avoid critical design flaws. Unfortunately, functional requirements usually get in the way of the security features, and the development team may not correctly ad…
▽ More
Secure by Design has become the mainstream development approach ensuring that software systems are not vulnerable to cyberattacks. Architectural security controls need to be carefully monitored over the software development life cycle to avoid critical design flaws. Unfortunately, functional requirements usually get in the way of the security features, and the development team may not correctly address critical security requirements. Identifying tactic-related code pieces in a software project enables an efficient review of the security controls' implementation as well as a resilient software architecture. This paper enumerates a comprehensive list of commonly used security controls and creates a dataset for each one of them by pulling related and unrelated code snippets from the open API of the StackOverflow question and answer platform. It uses the state-of-the-art NLP technique Bidirectional Encoder Representations from Transformers (BERT) and the Tactic Detector from our prior work to show that code pieces that implement security controls could be identified with high confidence. The results show that our model trained on tactic-related and unrelated code snippets derived from StackOverflow is able to identify tactic-related code pieces with F-Measure values above 0.9.
△ Less
Submitted 10 July, 2023;
originally announced July 2023.
