Safe Reinforcement Learning for Sepsis Treatment

Yan Jia; John Burden; Tom Lawton; Ibrahim Habli

Safe Reinforcement Learning for Sepsis Treatment

Yan Jia, John Burden, Tom Lawton, Ibrahim Habli

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed, at an early stage, and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn the optimal strategy to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with current clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated it to assess whether it has this kind of sudden major change when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.

Original language	English
Title of host publication	8th IEEE International Conference on Healthcare Informatics
Publication status	Published - 30 Nov 2020

Access to Document

ICHI_2020_paper_184
This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details.
Accepted author manuscript, 539 KB

Cite this

@inproceedings{b7954861954d4eedab85774f100c1410,

title = "Safe Reinforcement Learning for Sepsis Treatment",

abstract = "Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed, at an early stage, and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn the optimal strategy to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with current clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated it to assess whether it has this kind of sudden major change when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.",

author = "Yan Jia and John Burden and Tom Lawton and Ibrahim Habli",

year = "2020",

month = nov,

day = "30",

language = "English",

booktitle = "8th IEEE International Conference on Healthcare Informatics",

}

TY - GEN

T1 - Safe Reinforcement Learning for Sepsis Treatment

AU - Jia, Yan

AU - Burden, John

AU - Lawton, Tom

AU - Habli, Ibrahim

PY - 2020/11/30

Y1 - 2020/11/30

N2 - Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed, at an early stage, and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn the optimal strategy to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with current clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated it to assess whether it has this kind of sudden major change when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.

AB - Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed, at an early stage, and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn the optimal strategy to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with current clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated it to assess whether it has this kind of sudden major change when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.

M3 - Conference contribution

BT - 8th IEEE International Conference on Healthcare Informatics

ER -