-
An Agent-based Approach to Automated Game Testing: an Experience Report
Authors:
I. S. W. B. Prasetya,
Fernando Pastor Ricós,
Fitsum Kifetew,
Davide Prandi,
Samira Shirzadeh-hajimahmood,
Tanja E. J. Vos,
Premysl Paska,
Karel Hovorska,
Raihana Ferdous,
Angelo Susi,
Joseph Davidson
Abstract:
Computer games are very challenging to handle for traditional automated testing algorithms. In this paper we will look at intelligent agents as a solution. Agents are suitable for testing games, since they are reactive and able to reason about their environment to decide the action they want to take. This paper presents the experience of using an agent-based automated testing framework called \ivx…
▽ More
Computer games are very challenging to handle for traditional automated testing algorithms. In this paper we will look at intelligent agents as a solution. Agents are suitable for testing games, since they are reactive and able to reason about their environment to decide the action they want to take. This paper presents the experience of using an agent-based automated testing framework called \ivxr\ to test computer games. Three games will be discussed, including a sophisticated 3D game called Space Engineers. We will show how the framework can be used in different ways, either directly to drive a test agent, or as an intelligent functionality that can be driven by a traditional automated testing algorithm such as a random algorithm or a model based testing algorithm.
△ Less
Submitted 11 November, 2022;
originally announced November 2022.
-
Collaborative Artificial Intelligence Needs Stronger Assurances Driven by Risks
Authors:
Jubril Gbolahan Adigun,
Matteo Camilli,
Michael Felderer,
Andrea Giusti,
Dominik T Matt,
Anna Perini,
Barbara Russo,
Angelo Susi
Abstract:
Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for…
▽ More
Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for such systems since much work remains to manage possible risks. We identify emerging problems in this context and then we report our vision, as well as the progress of our multidisciplinary research team composed of software/systems, and mechatronics engineers to develop a risk-driven assurance process for CAISs.
△ Less
Submitted 22 September, 2022; v1 submitted 1 December, 2021;
originally announced December 2021.
-
Towards Risk Modeling for Collaborative AI
Authors:
Matteo Camilli,
Michael Felderer,
Andrea Giusti,
Dominik T. Matt,
Anna Perini,
Barbara Russo,
Angelo Susi
Abstract:
Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements domain specific standards and regulations is of greatest importance. Challenges associated with the achieveme…
▽ More
Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements domain specific standards and regulations is of greatest importance. Challenges associated with the achievement of this goal become even more severe when such systems rely on machine learning components rather than such as top-down rule-based AI. In this paper, we introduce a risk modeling approach tailored to Collaborative AI systems. The risk model includes goals, risk events and domain specific indicators that potentially expose humans to hazards. The risk model is then leveraged to drive assurance methods that feed in turn the risk model through insights extracted from run-time evidence. Our envisioned approach is described by means of a running example in the domain of Industry 4.0, where a robotic arm endowed with a visual perception component, implemented with machine learning, collaborates with a human operator for a production-relevant task.
△ Less
Submitted 12 March, 2021;
originally announced March 2021.
-
Formalization and Validation of Safety-Critical Requirements
Authors:
Alessandro Cimatti,
Marco Roveri,
Angelo Susi,
Stefano Tonetta
Abstract:
The validation of requirements is a fundamental step in the development process of safety-critical systems. In safety critical applications such as aerospace, avionics and railways, the use of formal methods is of paramount importance both for requirements and for design validation. Nevertheless, while for the verification of the design, many formal techniques have been conceived and applied, the…
▽ More
The validation of requirements is a fundamental step in the development process of safety-critical systems. In safety critical applications such as aerospace, avionics and railways, the use of formal methods is of paramount importance both for requirements and for design validation. Nevertheless, while for the verification of the design, many formal techniques have been conceived and applied, the research on formal methods for requirements validation is not yet mature. The main obstacles are that, on the one hand, the correctness of requirements is not formally defined; on the other hand that the formalization and the validation of the requirements usually demands a strong involvement of domain experts. We report on a methodology and a series of techniques that we developed for the formalization and validation of high-level requirements for safety-critical applications. The main ingredients are a very expressive formal language and automatic satisfiability procedures. The language combines first-order, temporal, and hybrid logic. The satisfiability procedures are based on model checking and satisfiability modulo theory. We applied this technology within an industrial project to the validation of railways requirements.
△ Less
Submitted 27 June, 2012; v1 submitted 8 March, 2010;
originally announced March 2010.
-
Theory of Regulatory Compliance for Requirements Engineering
Authors:
Ivan Jureta,
Alberto Siena,
John Mylopoulos,
Anna Perini,
Angelo Susi
Abstract:
Regulatory compliance is increasingly being addressed in the practice of requirements engineering as a main stream concern. This paper points out a gap in the theoretical foundations of regulatory compliance, and presents a theory that states (i) what it means for requirements to be compliant, (ii) the compliance problem, i.e., the problem that the engineer should resolve in order to verify whet…
▽ More
Regulatory compliance is increasingly being addressed in the practice of requirements engineering as a main stream concern. This paper points out a gap in the theoretical foundations of regulatory compliance, and presents a theory that states (i) what it means for requirements to be compliant, (ii) the compliance problem, i.e., the problem that the engineer should resolve in order to verify whether requirements are compliant, and (iii) testable hypotheses (predictions) about how compliance of requirements is verified. The theory is instantiated by presenting a requirements engineering framework that implements its principles, and is exemplified on a real-world case study.
△ Less
Submitted 19 February, 2010;
originally announced February 2010.
