Deep Residual Compensation Convolutional Network without Backpropagation

Mubarakah Alotaibi; Richard Charles Wilson

Deep Residual Compensation Convolutional Network without Backpropagation

Mubarakah Alotaibi^*, Richard Charles Wilson

^*Corresponding author for this work

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

PCANet and its variants provided good accuracy
results for classification tasks. However, despite the importance
of network depth in achieving good classification accuracy, these
networks were trained with a maximum of nine layers. In
this paper, we introduce a residual compensation convolutional
network, which is the first PCANet-like network trained with
hundreds of layers while improving classification accuracy. The
design of the proposed network consists of several convolutional
layers, each followed by post-processing steps and a classifier.
To correct the classification errors and significantly increase the
network’s depth, we train each layer with new labels derived from
the residual information of all its preceding layers. This learning
mechanism is accomplished by traversing the network’s layers
in a single forward pass without backpropagation or gradient
computations. Our experiments on four distinct classification
benchmarks (MNIST, CIFAR-10, CIFAR-100, and TinyImageNet)
show that our deep network outperforms all existing
PCANet-like networks and is competitive with several traditional
gradient-based models.

Original language	English
Title of host publication	International Joint Conference on Neural Networks
Publisher	IEEE Computer Society
Publication status	Published - 18 Jun 2023

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

Access to Document

m81765-alotaibi final
This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details
Accepted author manuscript, 435 KB

Cite this

@inproceedings{04143d1feb0c4d059568fcedd34b73d8,

title = "Deep Residual Compensation Convolutional Network without Backpropagation",

abstract = "PCANet and its variants provided good accuracyresults for classification tasks. However, despite the importanceof network depth in achieving good classification accuracy, thesenetworks were trained with a maximum of nine layers. Inthis paper, we introduce a residual compensation convolutionalnetwork, which is the first PCANet-like network trained withhundreds of layers while improving classification accuracy. Thedesign of the proposed network consists of several convolutionallayers, each followed by post-processing steps and a classifier.To correct the classification errors and significantly increase thenetwork{\textquoteright}s depth, we train each layer with new labels derived fromthe residual information of all its preceding layers. This learningmechanism is accomplished by traversing the network{\textquoteright}s layersin a single forward pass without backpropagation or gradientcomputations. Our experiments on four distinct classificationbenchmarks (MNIST, CIFAR-10, CIFAR-100, and TinyImageNet)show that our deep network outperforms all existingPCANet-like networks and is competitive with several traditionalgradient-based models.",

author = "Mubarakah Alotaibi and Wilson, {Richard Charles}",

note = "This is an author-produced version of the published paper. Uploaded in accordance with the publisher{\textquoteright}s self-archiving policy. Further copying may not be permitted; contact the publisher for details",

year = "2023",

month = jun,

day = "18",

language = "English",

booktitle = "International Joint Conference on Neural Networks",

publisher = "IEEE Computer Society",

address = "United States",

}

TY - GEN

T1 - Deep Residual Compensation Convolutional Network without Backpropagation

AU - Alotaibi, Mubarakah

AU - Wilson, Richard Charles

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

PY - 2023/6/18

Y1 - 2023/6/18

N2 - PCANet and its variants provided good accuracyresults for classification tasks. However, despite the importanceof network depth in achieving good classification accuracy, thesenetworks were trained with a maximum of nine layers. Inthis paper, we introduce a residual compensation convolutionalnetwork, which is the first PCANet-like network trained withhundreds of layers while improving classification accuracy. Thedesign of the proposed network consists of several convolutionallayers, each followed by post-processing steps and a classifier.To correct the classification errors and significantly increase thenetwork’s depth, we train each layer with new labels derived fromthe residual information of all its preceding layers. This learningmechanism is accomplished by traversing the network’s layersin a single forward pass without backpropagation or gradientcomputations. Our experiments on four distinct classificationbenchmarks (MNIST, CIFAR-10, CIFAR-100, and TinyImageNet)show that our deep network outperforms all existingPCANet-like networks and is competitive with several traditionalgradient-based models.

AB - PCANet and its variants provided good accuracyresults for classification tasks. However, despite the importanceof network depth in achieving good classification accuracy, thesenetworks were trained with a maximum of nine layers. Inthis paper, we introduce a residual compensation convolutionalnetwork, which is the first PCANet-like network trained withhundreds of layers while improving classification accuracy. Thedesign of the proposed network consists of several convolutionallayers, each followed by post-processing steps and a classifier.To correct the classification errors and significantly increase thenetwork’s depth, we train each layer with new labels derived fromthe residual information of all its preceding layers. This learningmechanism is accomplished by traversing the network’s layersin a single forward pass without backpropagation or gradientcomputations. Our experiments on four distinct classificationbenchmarks (MNIST, CIFAR-10, CIFAR-100, and TinyImageNet)show that our deep network outperforms all existingPCANet-like networks and is competitive with several traditionalgradient-based models.

M3 - Conference contribution

BT - International Joint Conference on Neural Networks

PB - IEEE Computer Society

ER -