A Bayesian Analysis of Algorithms for Learning Finite Functions

James Cussens

A Bayesian Analysis of Algorithms for Learning Finite Functions

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We consider algorithms for learning functions $f: X rightarrow Y$, where $X$ and $Y$ are finite, and there is assumed to be no noise in the data. Learning algorithms, , are connected with the set of prior probability distributions for which they are optimal. A method for constructing from is given and the relationship between the various is discussed. em Improper algorithms are identified as those for which has zero volume. Improper algorithms are investigated using linear algebra and two examples of improper algorithms are given. This framework is then applied to the question of choosing between competing algorithms. ``Leave-one-out'' cross-validation is hence characterised as a crude method of ML-II prior selection. We conclude by examining how the mathematical results bear on practical problems and by discussing related work, as well as suggesting future work.

Original language	English
Title of host publication	Machine Learning: Proceedings of the Twelfth International Conference (ML95)
Editors	Armand Prieditis, Stuart Russell
Place of Publication	San Francisco, CA
Publisher	Morgan Kaufmann Publishers
Pages	142-149
Number of pages	8
Publication status	Published - 1995

Access to Document

ftp://ftp.cs.york.ac.uk/pub/aig/Papers/james.cussens/ml95.ps.gz

Cite this

@inproceedings{e11b244eec3d40f28b6c7af5d5f39613,

title = "A Bayesian Analysis of Algorithms for Learning Finite Functions",

abstract = "We consider algorithms for learning functions $f: X rightarrow Y$, where $X$ and $Y$ are finite, and there is assumed to be no noise in the data. Learning algorithms, , are connected with the set of prior probability distributions for which they are optimal. A method for constructing from is given and the relationship between the various is discussed. em Improper algorithms are identified as those for which has zero volume. Improper algorithms are investigated using linear algebra and two examples of improper algorithms are given. This framework is then applied to the question of choosing between competing algorithms. ``Leave-one-out'' cross-validation is hence characterised as a crude method of ML-II prior selection. We conclude by examining how the mathematical results bear on practical problems and by discussing related work, as well as suggesting future work.",

author = "James Cussens",

year = "1995",

language = "English",

pages = "142--149",

editor = "Armand Prieditis and Stuart Russell",

booktitle = "Machine Learning: Proceedings of the Twelfth International Conference (ML95)",

publisher = "Morgan Kaufmann Publishers",

}

TY - GEN

T1 - A Bayesian Analysis of Algorithms for Learning Finite Functions

AU - Cussens, James

PY - 1995

Y1 - 1995

N2 - We consider algorithms for learning functions $f: X rightarrow Y$, where $X$ and $Y$ are finite, and there is assumed to be no noise in the data. Learning algorithms, , are connected with the set of prior probability distributions for which they are optimal. A method for constructing from is given and the relationship between the various is discussed. em Improper algorithms are identified as those for which has zero volume. Improper algorithms are investigated using linear algebra and two examples of improper algorithms are given. This framework is then applied to the question of choosing between competing algorithms. ``Leave-one-out'' cross-validation is hence characterised as a crude method of ML-II prior selection. We conclude by examining how the mathematical results bear on practical problems and by discussing related work, as well as suggesting future work.

AB - We consider algorithms for learning functions $f: X rightarrow Y$, where $X$ and $Y$ are finite, and there is assumed to be no noise in the data. Learning algorithms, , are connected with the set of prior probability distributions for which they are optimal. A method for constructing from is given and the relationship between the various is discussed. em Improper algorithms are identified as those for which has zero volume. Improper algorithms are investigated using linear algebra and two examples of improper algorithms are given. This framework is then applied to the question of choosing between competing algorithms. ``Leave-one-out'' cross-validation is hence characterised as a crude method of ML-II prior selection. We conclude by examining how the mathematical results bear on practical problems and by discussing related work, as well as suggesting future work.

M3 - Conference contribution

SP - 142

EP - 149

BT - Machine Learning: Proceedings of the Twelfth International Conference (ML95)

A2 - Prieditis, Armand

A2 - Russell, Stuart

PB - Morgan Kaufmann Publishers

CY - San Francisco, CA

ER -