Unsupervised learning of word segmentation rules with genetic algorithms and inductive logic programming

D Kazakov; S Manandhar

doi:10.1023/A:1007629103294

Unsupervised learning of word segmentation rules with genetic algorithms and inductive logic programming

D Kazakov, S Manandhar

Computer Science

Research output: Contribution to journal › Article › peer-review

Abstract

This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list of words. First, a language bias for word se mentation is introduced and a simple genetic algorithm is used in the search for a segmentation that corresponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision list learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.

Original language	English
Pages (from-to)	121-162
Number of pages	42
Journal	Machine Learning
Volume	43
Issue number	1-2
DOIs	https://doi.org/10.1023/A:1007629103294
Publication status	Published - Apr 2001

Keywords

unsupervised machine learning
inductive logic programming
natural language
word segmentation

Access to Document

10.1023/A:1007629103294

http://dx.doi.org/10.1023/A:1007629103294

Cite this

@article{1c9187e2be064062bee71ab6f31e8bb0,

title = "Unsupervised learning of word segmentation rules with genetic algorithms and inductive logic programming",

abstract = "This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list of words. First, a language bias for word se mentation is introduced and a simple genetic algorithm is used in the search for a segmentation that corresponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision list learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.",

keywords = "unsupervised machine learning, inductive logic programming, natural language, word segmentation",

author = "D Kazakov and S Manandhar",

year = "2001",

month = apr,

doi = "10.1023/A:1007629103294",

language = "English",

volume = "43",

pages = "121--162",

journal = "Machine Learning",

issn = "0885-6125",

publisher = "Springer",

number = "1-2",

}

TY - JOUR

T1 - Unsupervised learning of word segmentation rules with genetic algorithms and inductive logic programming

AU - Kazakov, D

AU - Manandhar, S

PY - 2001/4

Y1 - 2001/4

N2 - This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list of words. First, a language bias for word se mentation is introduced and a simple genetic algorithm is used in the search for a segmentation that corresponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision list learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.

AB - This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list of words. First, a language bias for word se mentation is introduced and a simple genetic algorithm is used in the search for a segmentation that corresponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision list learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.

KW - unsupervised machine learning

KW - inductive logic programming

KW - natural language

KW - word segmentation

U2 - 10.1023/A:1007629103294

DO - 10.1023/A:1007629103294

M3 - Article

SN - 0885-6125

VL - 43

SP - 121

EP - 162

JO - Machine Learning

JF - Machine Learning

IS - 1-2

ER -