Test and Evaluation Framework for Multi-Agent Systems of Autonomous Intelligent Agents
Authors:
Erin Lanus,
Ivan Hernandez,
Adam Dachowicz,
Laura Freeman,
Melanie Grande,
Andrew Lang,
Jitesh H. Panchal,
Anthony Patrick,
Scott Welch
Abstract:
Test and evaluation is a necessary process for ensuring that engineered systems perform as intended under a variety of conditions, both expected and unexpected. In this work, we consider the unique challenges of develo** a unifying test and evaluation framework for complex ensembles of cyber-physical systems with embedded artificial intelligence. We propose a framework that incorporates test and…
▽ More
Test and evaluation is a necessary process for ensuring that engineered systems perform as intended under a variety of conditions, both expected and unexpected. In this work, we consider the unique challenges of develo** a unifying test and evaluation framework for complex ensembles of cyber-physical systems with embedded artificial intelligence. We propose a framework that incorporates test and evaluation throughout not only the development life cycle, but continues into operation as the system learns and adapts in a noisy, changing, and contended environment. The framework accounts for the challenges of testing the integration of diverse systems at various hierarchical scales of composition while respecting that testing time and resources are limited. A generic use case is provided for illustrative purposes and research directions emerging as a result of exploring the use case via the framework are suggested.
△ Less
Submitted 25 January, 2021;
originally announced January 2021.
