By the same authors

Importance-Driven Deep Learning System Testing

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Full text download(s)

Author(s)

Department/unit(s)

Publication details

Title of host publication42nd International Conference on Software Engineering
DateAccepted/In press - 9 Dec 2019
DatePublished (current) - 2020
Original languageEnglish

Abstract

Deep Learning (DL) systems are key enablers for engineering intelligent applications due to their ability to solve complex tasks such as image recognition and machine translation. Nevertheless, using DL systems in safety- and security-critical applications requires to provide testing evidence for their dependable operation. Recent research in this direction focuses on adapting testing criteria from traditional software engineering as a means of increasing confidence for their correct behaviour. However, they are inadequate in capturing the intrinsic properties exhibited by these systems. We bridge this gap by introducing DeepImportance, a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion for DL systems. Applying IDC enables to establish a layer-wise functional understanding of the importance of DL system components and use this information to guide the generation of semantically-diverse test sets. Our empirical evaluation on several DL systems, across multiple DL datasets and with state-of-the-art adversarial generation techniques demonstrates the usefulness and effectiveness of DeepImportance and its ability to guide the engineering of more robust DL systems.

Discover related content

Find related publications, people, projects, datasets and more using interactive charts.

View graph of relations