Image Adaptation for Colour Vision Deficient Viewers Using Vision Transformers

TY - GEN

T1 - Image Adaptation for Colour Vision Deficient Viewers Using Vision Transformers

AU - Gilooly, Thomas

AU - Thomas, Jean-Baptiste

AU - Hardeberg, Jon Yngve

AU - Guarnera, Giuseppe Claudio

PY - 2024/10/28

Y1 - 2024/10/28

N2 - Colour Vision Deficiency (CVD) occurs when anomalous retinal cone spectral responses impact the ability to distinguish between certain colours. To enhance image quality and viewing experience, recolouring algorithms seek to modify pixel values so that this does not lead to a loss of detail or image quality. Recent approaches to recolouring for CVD viewers employ neural models which exploit higher order features to direct colour adaptation. In this work, we build upon the idea that visual neural models exhibit emergent behaviour which mimics the human visual system. We make use of these learned behaviours to guide the colour adaptation process by considering regions of the image that are the most semantically meaningful for a non-CVD viewer and compensate for them appropriately if they are absent or distorted in a CVD-simulated version of the image. We find that a minimal algorithm built atop a pre-trained model produces results that substantially boost contrast and saliencefor viewers affected by CVD. We also investigate a few cases where modifications are absent, indicating that a neurally guided salience-based model may also provide a means ofdetermining when recolouring is not necessary. Additionally, we introduce a novel metric that quantifies the contrast increase or decrease under changes in image colour.

AB - Colour Vision Deficiency (CVD) occurs when anomalous retinal cone spectral responses impact the ability to distinguish between certain colours. To enhance image quality and viewing experience, recolouring algorithms seek to modify pixel values so that this does not lead to a loss of detail or image quality. Recent approaches to recolouring for CVD viewers employ neural models which exploit higher order features to direct colour adaptation. In this work, we build upon the idea that visual neural models exhibit emergent behaviour which mimics the human visual system. We make use of these learned behaviours to guide the colour adaptation process by considering regions of the image that are the most semantically meaningful for a non-CVD viewer and compensate for them appropriately if they are absent or distorted in a CVD-simulated version of the image. We find that a minimal algorithm built atop a pre-trained model produces results that substantially boost contrast and saliencefor viewers affected by CVD. We also investigate a few cases where modifications are absent, indicating that a neurally guided salience-based model may also provide a means ofdetermining when recolouring is not necessary. Additionally, we introduce a novel metric that quantifies the contrast increase or decrease under changes in image colour.

M3 - Conference contribution

BT - IEEE/CVF Winter Conference on Applications of Computer Vision 2025

T2 - IEEE/CVF Winter Conference on Applications of Computer Vision 2025

Y2 - 28 February 2025 through 4 March 2026

ER -