Penalizing small errors using an Adaptive Logarithmic Loss

Chaitanya Kaul; Nicholas Edwin Pears; Hang Dai; Roderick  Murray-Smith; Suresh Manandhar

Penalizing small errors using an Adaptive Logarithmic Loss

Chaitanya Kaul, Nicholas Edwin Pears, Hang Dai, Roderick Murray-Smith, Suresh Manandhar

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Loss functions are error metrics that quantify the difference between a prediction and its corresponding ground truth. Fundamentally, they define a functional landscape for traversal by gradient descent. Although numerous loss functions have been proposed to date in order to handle various machine learning problems, little attention has been given to enhancing these functions to better traverse the loss landscape. In this paper, we simultaneously and significantly mitigate two prominent problems in medical image segmentation namely: i) class imbalance between foreground and background pixels and ii) poor loss function convergence. To this end, we propose an adaptive logarithmic loss function. We compare this loss function with the existing state-of-the-art on the ISIC 2018 dataset, the nuclei segmentation dataset as well as the DRIVE retinal vessel segmentation dataset. We measure the performance of our methodology on benchmark metrics and demonstrate state-of-the-art performance. More generally, we show that our system can be used as a framework for better training of deep neural networks.

Original language	English
Title of host publication	25th International Conference on Pattern Recognition
Subtitle of host publication	AIHA-2020 – ICPR International Workshop on Artificial Intelligence for Healthcare Applications
Publisher	Springer
Publication status	Published - 10 Jan 2021
Event	25th International Conference on Pattern Recognition - Milan, Italy Duration: 10 Jan 2021 → 15 Jan 2021 https://www.micc.unifi.it/icpr2020/

Conference

Conference	25th International Conference on Pattern Recognition
Abbreviated title	ICPR 2020
Country/Territory	Italy
City	Milan
Period	10/01/21 → 15/01/21
Internet address	https://www.micc.unifi.it/icpr2020/

Cite this

@inproceedings{ce4b4f4ba9844319a62b4048bf99edd7,

title = "Penalizing small errors using an Adaptive Logarithmic Loss",

abstract = "Loss functions are error metrics that quantify the difference between a prediction and its corresponding ground truth. Fundamentally, they define a functional landscape for traversal by gradient descent. Although numerous loss functions have been proposed to date in order to handle various machine learning problems, little attention has been given to enhancing these functions to better traverse the loss landscape. In this paper, we simultaneously and significantly mitigate two prominent problems in medical image segmentation namely: i) class imbalance between foreground and background pixels and ii) poor loss function convergence. To this end, we propose an adaptive logarithmic loss function. We compare this loss function with the existing state-of-the-art on the ISIC 2018 dataset, the nuclei segmentation dataset as well as the DRIVE retinal vessel segmentation dataset. We measure the performance of our methodology on benchmark metrics and demonstrate state-of-the-art performance. More generally, we show that our system can be used as a framework for better training of deep neural networks.",

author = "Chaitanya Kaul and Pears, {Nicholas Edwin} and Hang Dai and Roderick Murray-Smith and Suresh Manandhar",

year = "2021",

month = jan,

day = "10",

language = "English",

booktitle = "25th International Conference on Pattern Recognition",

publisher = "Springer",

address = "Germany",

note = "25th International Conference on Pattern Recognition, ICPR 2020 ; Conference date: 10-01-2021 Through 15-01-2021",

url = "https://www.micc.unifi.it/icpr2020/",

}

Penalizing small errors using an Adaptive Logarithmic Loss. / Kaul, Chaitanya; Pears, Nicholas Edwin; Dai, Hang et al.
25th International Conference on Pattern Recognition: AIHA-2020 – ICPR International Workshop on Artificial Intelligence for Healthcare Applications. Springer, 2021.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Penalizing small errors using an Adaptive Logarithmic Loss

AU - Kaul, Chaitanya

AU - Pears, Nicholas Edwin

AU - Dai, Hang

AU - Murray-Smith, Roderick

AU - Manandhar, Suresh

PY - 2021/1/10

Y1 - 2021/1/10

N2 - Loss functions are error metrics that quantify the difference between a prediction and its corresponding ground truth. Fundamentally, they define a functional landscape for traversal by gradient descent. Although numerous loss functions have been proposed to date in order to handle various machine learning problems, little attention has been given to enhancing these functions to better traverse the loss landscape. In this paper, we simultaneously and significantly mitigate two prominent problems in medical image segmentation namely: i) class imbalance between foreground and background pixels and ii) poor loss function convergence. To this end, we propose an adaptive logarithmic loss function. We compare this loss function with the existing state-of-the-art on the ISIC 2018 dataset, the nuclei segmentation dataset as well as the DRIVE retinal vessel segmentation dataset. We measure the performance of our methodology on benchmark metrics and demonstrate state-of-the-art performance. More generally, we show that our system can be used as a framework for better training of deep neural networks.

AB - Loss functions are error metrics that quantify the difference between a prediction and its corresponding ground truth. Fundamentally, they define a functional landscape for traversal by gradient descent. Although numerous loss functions have been proposed to date in order to handle various machine learning problems, little attention has been given to enhancing these functions to better traverse the loss landscape. In this paper, we simultaneously and significantly mitigate two prominent problems in medical image segmentation namely: i) class imbalance between foreground and background pixels and ii) poor loss function convergence. To this end, we propose an adaptive logarithmic loss function. We compare this loss function with the existing state-of-the-art on the ISIC 2018 dataset, the nuclei segmentation dataset as well as the DRIVE retinal vessel segmentation dataset. We measure the performance of our methodology on benchmark metrics and demonstrate state-of-the-art performance. More generally, we show that our system can be used as a framework for better training of deep neural networks.

M3 - Conference contribution

BT - 25th International Conference on Pattern Recognition

PB - Springer

T2 - 25th International Conference on Pattern Recognition

Y2 - 10 January 2021 through 15 January 2021

ER -