Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

Simon Burton; Lydia Gauerhof; Richard David Hawkins; Ibrahim Habli; Bibhuti  Sethy

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

Simon Burton, Lydia Gauerhof, Richard David Hawkins, Ibrahim Habli, Bibhuti Sethy

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Due to their ability to efficiently process unstructured and highly dimensional input data, machine learning algorithms are being applied to perception tasks for highly automated driving functions. The consequences of failures and insu_ciencies in such algorithms are severe and a convincing assurance case that the algorithms meet certain safety requirements is therefore required. However, the task of demonstrating the performance of such algorithms is non-trivial, and as yet, no consensus has formed regarding an appropriate set of verification measures. This paper provides a framework for reasoning about the contribution of performance evidence to the assurance case for machine learning in an automated driving context and applies the evaluation criteria to a pedestrian recognition case study.

Original language	English
Title of host publication	Safecomp 2019 - Workshop on Artificial Intelligence Safety Engineering (Waise) of the 38th International Conference on Computer Safety, Reliability and Security
Publication status	Accepted/In press - 6 Jun 2019

Access to Document

WAISE2019Accepted author manuscript, 3.79 MB

Cite this

@inproceedings{dae0399a6b964aca80e716394c8ba172,

title = "Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions",

abstract = "Due to their ability to efficiently process unstructured and highly dimensional input data, machine learning algorithms are being applied to perception tasks for highly automated driving functions. The consequences of failures and insu_ciencies in such algorithms are severe and a convincing assurance case that the algorithms meet certain safety requirements is therefore required. However, the task of demonstrating the performance of such algorithms is non-trivial, and as yet, no consensus has formed regarding an appropriate set of verification measures. This paper provides a framework for reasoning about the contribution of performance evidence to the assurance case for machine learning in an automated driving context and applies the evaluation criteria to a pedestrian recognition case study.",

author = "Simon Burton and Lydia Gauerhof and Hawkins, {Richard David} and Ibrahim Habli and Bibhuti Sethy",

year = "2019",

month = jun,

day = "6",

language = "English",

booktitle = "Safecomp 2019 - Workshop on Artificial Intelligence Safety Engineering (Waise) of the 38th International Conference on Computer Safety, Reliability and Security",

}

Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions. / Burton, Simon; Gauerhof, Lydia; Hawkins, Richard David et al.

Safecomp 2019 - Workshop on Artificial Intelligence Safety Engineering (Waise) of the 38th International Conference on Computer Safety, Reliability and Security. 2019.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Confidence Arguments for Evidence of Performance in Machine Learning for Highly Automated Driving Functions

AU - Burton, Simon

AU - Gauerhof, Lydia

AU - Hawkins, Richard David

AU - Habli, Ibrahim

AU - Sethy, Bibhuti

PY - 2019/6/6

Y1 - 2019/6/6

N2 - Due to their ability to efficiently process unstructured and highly dimensional input data, machine learning algorithms are being applied to perception tasks for highly automated driving functions. The consequences of failures and insu_ciencies in such algorithms are severe and a convincing assurance case that the algorithms meet certain safety requirements is therefore required. However, the task of demonstrating the performance of such algorithms is non-trivial, and as yet, no consensus has formed regarding an appropriate set of verification measures. This paper provides a framework for reasoning about the contribution of performance evidence to the assurance case for machine learning in an automated driving context and applies the evaluation criteria to a pedestrian recognition case study.

AB - Due to their ability to efficiently process unstructured and highly dimensional input data, machine learning algorithms are being applied to perception tasks for highly automated driving functions. The consequences of failures and insu_ciencies in such algorithms are severe and a convincing assurance case that the algorithms meet certain safety requirements is therefore required. However, the task of demonstrating the performance of such algorithms is non-trivial, and as yet, no consensus has formed regarding an appropriate set of verification measures. This paper provides a framework for reasoning about the contribution of performance evidence to the assurance case for machine learning in an automated driving context and applies the evaluation criteria to a pedestrian recognition case study.

M3 - Conference contribution

BT - Safecomp 2019 - Workshop on Artificial Intelligence Safety Engineering (Waise) of the 38th International Conference on Computer Safety, Reliability and Security

ER -