Continual Unsupervised Generative Modeling

Fei Ye; Adrian Gheorghe Bors

doi:10.1109/TPAMI.2025.3564188

Continual Unsupervised Generative Modeling

Fei Ye, Adrian Gheorghe Bors

Computer Science

Research output: Contribution to journal › Article › peer-review

Abstract

Variational Autoencoders (VAEs), can achieve remarkable results in single tasks learning of data representations, image generation, and image-to-image translation among others. However, VAEs suffer from loss of information when aiming to continuously learn a sequence of different data domains. These are caused by the catastrophic forgetting, which affects all machine learning methods. This paper addresses the problem of catastrophic forgetting by developing a new theoretical framework which derives an upper bound to the negative sample log-likelihood when continuously learning sequences of tasks. These theoretical derivations provide new insights into the forgetting behavior of networks, showing that their optimal performance is achieved when a dynamic mixture expansion model adds new components whenever learning new tasks. In our approach we optimize the model size by introducing the Dynamic Expansion Graph Model (DEGM) that dynamically builds a graph structure promoting the positive knowledge transfer when learning new tasks. In addition, we propose a Dynamic Expansion Graph Adaptive Mechanism (DEGAM) that generates adaptive weights to regulate the graph structure, further improving the positive knowledge transfer effectiveness. Experimental results show that the proposed methodology performs better than other baselines in continual learning.

Original language	English
Pages (from-to)	1-18
Number of pages	18
Journal	IEEE Transactions on Pattern Analysis and Machine Intelligence
DOIs	https://doi.org/10.1109/TPAMI.2025.3564188
Publication status	Published - 25 Apr 2025

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

Access to Document

10.1109/TPAMI.2025.3564188

DEGM-TPAMI25
This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.
Accepted author manuscript, 4.24 MBLicence: CC BY

Cite this

@article{f96772b0cc674bd7b3e8e6f17d0c7d1e,

title = "Continual Unsupervised Generative Modeling",

abstract = "Variational Autoencoders (VAEs), can achieve remarkable results in single tasks learning of data representations, image generation, and image-to-image translation among others. However, VAEs suffer from loss of information when aiming to continuously learn a sequence of different data domains. These are caused by the catastrophic forgetting, which affects all machine learning methods. This paper addresses the problem of catastrophic forgetting by developing a new theoretical framework which derives an upper bound to the negative sample log-likelihood when continuously learning sequences of tasks. These theoretical derivations provide new insights into the forgetting behavior of networks, showing that their optimal performance is achieved when a dynamic mixture expansion model adds new components whenever learning new tasks. In our approach we optimize the model size by introducing the Dynamic Expansion Graph Model (DEGM) that dynamically builds a graph structure promoting the positive knowledge transfer when learning new tasks. In addition, we propose a Dynamic Expansion Graph Adaptive Mechanism (DEGAM) that generates adaptive weights to regulate the graph structure, further improving the positive knowledge transfer effectiveness. Experimental results show that the proposed methodology performs better than other baselines in continual learning.",

author = "Fei Ye and Bors, {Adrian Gheorghe}",

note = "This is an author-produced version of the published paper. Uploaded in accordance with the University{\textquoteright}s Research Publications and Open Access policy. ",

year = "2025",

month = apr,

day = "25",

doi = "10.1109/TPAMI.2025.3564188",

language = "English",

pages = "1--18",

journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",

issn = "0162-8828",

publisher = "IEEE Computer Society",

}

TY - JOUR

T1 - Continual Unsupervised Generative Modeling

AU - Ye, Fei

AU - Bors, Adrian Gheorghe

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy.

PY - 2025/4/25

Y1 - 2025/4/25

N2 - Variational Autoencoders (VAEs), can achieve remarkable results in single tasks learning of data representations, image generation, and image-to-image translation among others. However, VAEs suffer from loss of information when aiming to continuously learn a sequence of different data domains. These are caused by the catastrophic forgetting, which affects all machine learning methods. This paper addresses the problem of catastrophic forgetting by developing a new theoretical framework which derives an upper bound to the negative sample log-likelihood when continuously learning sequences of tasks. These theoretical derivations provide new insights into the forgetting behavior of networks, showing that their optimal performance is achieved when a dynamic mixture expansion model adds new components whenever learning new tasks. In our approach we optimize the model size by introducing the Dynamic Expansion Graph Model (DEGM) that dynamically builds a graph structure promoting the positive knowledge transfer when learning new tasks. In addition, we propose a Dynamic Expansion Graph Adaptive Mechanism (DEGAM) that generates adaptive weights to regulate the graph structure, further improving the positive knowledge transfer effectiveness. Experimental results show that the proposed methodology performs better than other baselines in continual learning.

AB - Variational Autoencoders (VAEs), can achieve remarkable results in single tasks learning of data representations, image generation, and image-to-image translation among others. However, VAEs suffer from loss of information when aiming to continuously learn a sequence of different data domains. These are caused by the catastrophic forgetting, which affects all machine learning methods. This paper addresses the problem of catastrophic forgetting by developing a new theoretical framework which derives an upper bound to the negative sample log-likelihood when continuously learning sequences of tasks. These theoretical derivations provide new insights into the forgetting behavior of networks, showing that their optimal performance is achieved when a dynamic mixture expansion model adds new components whenever learning new tasks. In our approach we optimize the model size by introducing the Dynamic Expansion Graph Model (DEGM) that dynamically builds a graph structure promoting the positive knowledge transfer when learning new tasks. In addition, we propose a Dynamic Expansion Graph Adaptive Mechanism (DEGAM) that generates adaptive weights to regulate the graph structure, further improving the positive knowledge transfer effectiveness. Experimental results show that the proposed methodology performs better than other baselines in continual learning.

U2 - 10.1109/TPAMI.2025.3564188

DO - 10.1109/TPAMI.2025.3564188

M3 - Article

SN - 0162-8828

SP - 1

EP - 18

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

ER -