Safety engineering, role responsibility and lessons from the Uber ATG Tempe Accident

Philippa Mary Ryan Conmy; Zoe Porter; Ibrahim Habli; John Alexander McDermid

doi:10.1145/3597512.3599718

Safety engineering, role responsibility and lessons from the Uber ATG Tempe Accident

Philippa Mary Ryan Conmy, Zoe Porter, Ibrahim Habli, John Alexander McDermid

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Safety critical autonomous systems (SCAS) require a safety assurance case (SAC) to justify why they are considered acceptably safe to use, despite the residual risk associated with their operation. Reducing risk is an overarching principle of all safety critical systems development and operation. The SAC should demonstrate that the risk is tolerable and has been reduced as far as possible, through robust design and operational controls. As a SCAS may not have an operator, safety engineers have a more direct responsibility for operational decisions. Following an accident it may be useful to understand which engineering decisions causally contributed to it, and roles responsible for those decisions. This paper contains a review of how different senses of responsibility (role, moral, legal and causal) apply to SCAS engineering and operation. We use this to illustrate how considering role responsibility can help support a defensible SAC, and potentially improve system safety practice. Our findings are illustrated with an analysis the Uber/Tempe Arizona fatal collision accident report. We found that existing safety practice may not identify all role responsibilities in a way that supports causal safety analysis. This paper is intended for the whole TAS community, but with an emphasis on safety professionals.

Original language	English
Title of host publication	First International Symposium on Trustworthy Autonomous Systems (TAS '23)
Publisher	Association for Computing Machinery, Inc
Number of pages	10
ISBN (Print)	979-8-4007-0734-6
DOIs	https://doi.org/10.1145/3597512.3599718
Publication status	Published - 11 Jul 2023

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

Access to Document

10.1145/3597512.3599718

TAS_23_paper_9381_version_3_accepted
This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.
Accepted author manuscript, 738 KBLicence: CC BY

Cite this

@inproceedings{5365c80a27e04935bd9a39d43c9b8b72,

title = "Safety engineering, role responsibility and lessons from the Uber ATG Tempe Accident",

abstract = "Safety critical autonomous systems (SCAS) require a safety assurance case (SAC) to justify why they are considered acceptably safe to use, despite the residual risk associated with their operation. Reducing risk is an overarching principle of all safety critical systems development and operation. The SAC should demonstrate that the risk is tolerable and has been reduced as far as possible, through robust design and operational controls. As a SCAS may not have an operator, safety engineers have a more direct responsibility for operational decisions. Following an accident it may be useful to understand which engineering decisions causally contributed to it, and roles responsible for those decisions. This paper contains a review of how different senses of responsibility (role, moral, legal and causal) apply to SCAS engineering and operation. We use this to illustrate how considering role responsibility can help support a defensible SAC, and potentially improve system safety practice. Our findings are illustrated with an analysis the Uber/Tempe Arizona fatal collision accident report. We found that existing safety practice may not identify all role responsibilities in a way that supports causal safety analysis. This paper is intended for the whole TAS community, but with an emphasis on safety professionals.",

author = "{Ryan Conmy}, {Philippa Mary} and Zoe Porter and Ibrahim Habli and McDermid, {John Alexander}",

note = "This is an author-produced version of the published paper. Uploaded in accordance with the University{\textquoteright}s Research Publications and Open Access policy.",

year = "2023",

month = jul,

day = "11",

doi = "10.1145/3597512.3599718",

language = "English",

isbn = "979-8-4007-0734-6",

booktitle = "First International Symposium on Trustworthy Autonomous Systems (TAS '23)",

publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Safety engineering, role responsibility and lessons from the Uber ATG Tempe Accident

AU - Ryan Conmy, Philippa Mary

AU - Porter, Zoe

AU - Habli, Ibrahim

AU - McDermid, John Alexander

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

PY - 2023/7/11

Y1 - 2023/7/11

N2 - Safety critical autonomous systems (SCAS) require a safety assurance case (SAC) to justify why they are considered acceptably safe to use, despite the residual risk associated with their operation. Reducing risk is an overarching principle of all safety critical systems development and operation. The SAC should demonstrate that the risk is tolerable and has been reduced as far as possible, through robust design and operational controls. As a SCAS may not have an operator, safety engineers have a more direct responsibility for operational decisions. Following an accident it may be useful to understand which engineering decisions causally contributed to it, and roles responsible for those decisions. This paper contains a review of how different senses of responsibility (role, moral, legal and causal) apply to SCAS engineering and operation. We use this to illustrate how considering role responsibility can help support a defensible SAC, and potentially improve system safety practice. Our findings are illustrated with an analysis the Uber/Tempe Arizona fatal collision accident report. We found that existing safety practice may not identify all role responsibilities in a way that supports causal safety analysis. This paper is intended for the whole TAS community, but with an emphasis on safety professionals.

AB - Safety critical autonomous systems (SCAS) require a safety assurance case (SAC) to justify why they are considered acceptably safe to use, despite the residual risk associated with their operation. Reducing risk is an overarching principle of all safety critical systems development and operation. The SAC should demonstrate that the risk is tolerable and has been reduced as far as possible, through robust design and operational controls. As a SCAS may not have an operator, safety engineers have a more direct responsibility for operational decisions. Following an accident it may be useful to understand which engineering decisions causally contributed to it, and roles responsible for those decisions. This paper contains a review of how different senses of responsibility (role, moral, legal and causal) apply to SCAS engineering and operation. We use this to illustrate how considering role responsibility can help support a defensible SAC, and potentially improve system safety practice. Our findings are illustrated with an analysis the Uber/Tempe Arizona fatal collision accident report. We found that existing safety practice may not identify all role responsibilities in a way that supports causal safety analysis. This paper is intended for the whole TAS community, but with an emphasis on safety professionals.

U2 - 10.1145/3597512.3599718

DO - 10.1145/3597512.3599718

M3 - Conference contribution

SN - 979-8-4007-0734-6

BT - First International Symposium on Trustworthy Autonomous Systems (TAS '23)

PB - Association for Computing Machinery, Inc

ER -