Robust Intent Classification using Bayesian LSTM for Clinical Conversational Agents (CAs)

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Conversational Agents (CAs) are software programs that replicate human conversations using machine learning (ML) and natural language processing (NLP). CAs are currently being utilised for diverse clinical applications such as symptom checking, health monitoring, medical triage and diagnosis. Intent classification (IC) is an essential task of understanding user utterance in CAs which makes use of modern deep learning (DL) methods. Because of the inherent model uncertainty associated with those methods, accuracy alone cannot be relied upon in clinical applications where certain errors may compromise patient safety. In this work, we employ Bayesian Long Short-Term Memory Networks (LSTMs) to calculate model uncertainty for IC, with a specific emphasis on symptom checker CAs. This method provides a certainty measure with IC prediction that can be utilised in assuring safe response from CAs. We evaluated our method on in-distribution (ID) and out-of-distribution (OOD) data and found mean uncer-tainty to be much higher for OOD data. These findings suggest that our method is robust to OOD utterances and can detect non-understanding errors in CAs.
Original languageEnglish
Title of host publicationMobiHealth 2021
Subtitle of host publicationWireless Mobile Communication and Healthcare
Place of PublicationCham
Number of pages13
ISBN (Electronic)9783031063688
ISBN (Print)9783031063671
Publication statusPublished - 7 Jun 2022
Event10th EAI International Conference on Wireless Mobile Communication and Healthcare - Chongqing, People’s Republic of China (online), Chongqing, China
Duration: 13 Nov 202114 Nov 2021


Conference10th EAI International Conference on Wireless Mobile Communication and Healthcare
Internet address

Bibliographical note

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details


  • Conversational Agents (CAs)
  • Machine Learning
  • Model Uncertainty
  • Out-of-Distribution (OOD)
  • Healthcare
  • Patient Safety

Cite this