[BibTeX] [RIS]
Learning rules for multi-label classification: a stacking and a separate-and-conquer approach
Type of publication: Article
Citation: loza16MLRL
Journal: Machine Learning
Volume: 105
Number: 1
Year: 2016
Month: May
Pages: 77--126
ISSN: 0885-6125
URL: https://www.ke.tu-darmstadt.de/publications/papers/loza16MLRL.pdf
DOI: 10.1007/s10994-016-5552-1
Abstract: Dependencies between the labels are commonly regarded as the crucial issue in multi-label classification. Rules provide a natural way for symbolically describing such relationships. For instance, rules with label tests in the body allow for representing directed dependencies like implications, subsumptions, or exclusions. Moreover, rules naturally allow to jointly capture both local and global label dependencies. In this paper, we introduce two approaches for learning such label-dependent rules. Our first solution is a bootstrapped stacking approach which can be built on top of a conventional rule learning algorithm. For this, we learn for each label a separate ruleset, but we include the remaining labels as additional attributes in the training instances. The second approach goes one step further by adapting the commonly used separate-and-conquer algorithm for learning multi-label rules. The main idea is to re-include the covered examples with the predicted labels so that this information can be used for learning subsequent rules. Both approaches allow for making label dependencies explicit in the rules. In addition, the usage of standard rule learning techniques targeted at producing accurate predictions ensures that the found rules are useful for actual classification. Our experiments show (a) that the discovered dependencies contribute to the understanding and improve the analysis of multi-label datasets, and (b) that the found multi-label rules are crucial for the predictive performance as our proposed approaches beat the baseline using conventional rules.
Keywords: Label Dependencies, multilabel classification, Rule Learning, Stacking
Authors Loza Mencía, Eneldo
Janssen, Frederik
Editors D{\v z}eroski, Sašo
Kocev, Dragi
Panov, Panče