By the same authors

Robust Intent Classification using Bayesian LSTM for Clinical Conversational Agents (CAs)

Research output: Contribution to conferencePaperpeer-review

Author(s)

Department/unit(s)

Publication details

DateAccepted/In press - 14 Nov 2021
Original languageEnglish

Abstract

Conversational Agents (CAs) are software programs that replicate hu-man conversations using machine learning (ML) and natural language processing (NLP). CAs are currently being utilised for diverse clinical applications such as symptom checking, health monitoring, medical triage and diagnosis. Intent clas-sification (IC) is an essential task of understanding user utterance in CAs which makes use of modern deep learning (DL) methods. Because of the inherent model uncertainty associated with those methods, accuracy alone cannot be relied upon
in clinical applications where certain errors may compromise patient safety. In this work, we employ Bayesian Long Short-Term Memory Networks (LSTMs) to calculate model uncertainty for IC, with a specific emphasis on symptom checker CAs. This method provides a certainty measure with IC prediction that can be utilised in assuring safe response from CAs. We evaluated our method on in-distribution (ID) and out-of-distribution (OOD) data and found mean uncer-tainty to be much higher for OOD data. These findings suggest that our method is robust to OOD utterances and can detect non-understanding errors in CAs.

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

    Research areas

  • Conversational Agents (CAs), Machine Learning, Model Uncertainty, Out-of-Distribution (OOD), Healthcare, Patient Safety

Discover related content

Find related publications, people, projects, datasets and more using interactive charts.

View graph of relations