Shifting paradigms in verification and validation of AI-enabled systems: A systems-theoretic perspective

Niloofar Shadab, Aditya U. Kulkarni, Alejandro Salado

Research output: Chapter in Book/Report/Conference proceedingChapter

2 Scopus citations

Abstract

There is a fundamental misalignment between current approaches to designing and executing verification and validation (V&V) strategies and the nature of AI-enabled systems. Current V&V approaches rely on the assumption that system behavior is preserved during a system's lifetime. However, AI-enabled systems are developed so that they evolve their own behavior during their lifetime; this is the consequence of learning by the AI-enabled system. This misalignment makes existing approaches to designing and executing V&V strategies ineffective. In this chapter, we will provide a systems-theoretic explanation for (1) why learning capabilities originate a unique and unprecedented family of systems, and (2) why current V&V methods and processes are not fit for purpose. AI-enabled systems necessitate a paradigm shift in V&V activities. To enable this shift, we will delineate a set of theoretical advances and process transformations that could support such shift.

Original languageEnglish (US)
Title of host publicationSystems Engineering and Artificial Intelligence
PublisherSpringer International Publishing
Pages363-378
Number of pages16
ISBN (Electronic)9783030772833
ISBN (Print)9783030772826
DOIs
StatePublished - Nov 2 2021

Keywords

  • AI-enabled systems
  • Cyber-physical systems
  • Systems engineering
  • Systems theory
  • Verification and validation

ASJC Scopus subject areas

  • Computer Science(all)

Cite this