Information technology — Artificial intelligence — Testing for AI systems — Part 11

Abstract

This document describes testing techniques (including those described in ISO/IEC/IEEE 29119-4) applicable for AI systems in the context of the AI system life cycle model stages defined in ISO/IEC 22989. It describes how AI and ML assessment metrics can be used in the context of those testing techniques. It also maps testing processes, including those described in ISO/IEC/IEEE 29119-2, to the verification and validation stages in the AI system life cycle. ©ISO/IEC 2022. All rigthts reserved.

Scope

What is ISO/IEC TR 29119-11 about?

ISO/IEC TR 29119-11 is an International Standard on software and systems engineering which is part 11 of a multi-series that provides an introduction to AI-based systems. These systems are typically complex (e.g., deep neural nets), are sometimes based on big data, can be poorly specified, and can be non-deterministic, which creates new challenges and opportunities for testing them.
ISO/IEC TR 29119-11 explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems.

Who is ISO/IEC TR 29119-11 for?

ISO/IEC TR 29119-11 on testing of AI-based systems is relevant to:

IT industry
Computer and data scientists
Machine learning engineers
Software engineers

Why should you use ISO/IEC TR 29119-11?

The testing of traditional systems is well-understood, but AI-based systems, which are becoming more prevalent and critical to our daily lives, introduce new challenges. ISO/IEC TR 29119-11 has been created to introduce AI-based systems and provide guidelines on how they might be tested.
ISO/IEC TR 29119-11 presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduce white-box testing specifically for neural networks. ISO/IEC TR 29119-11 describes options for the test environments and test scenarios used for testing AI-based systems.
Adopting the testing of AI-based systems in compliance with ISO/IEC TR 29119-11 can help overcome challenges related to systems specifications, test input, self-learning systems, and flexibility and adaptability, amongst others.© British Standards Institution 2022

External Links