Change with delayed labeling: when is it detectable?

Zliobaite, I., 2010. Change with delayed labeling: when is it detectable? In: The 5th International Workshop on Chance Discovery - IWCD5 in IEEE ICDM'10, pp. 843-850.

This is the latest version of this eprint.

Full text available as:

[img]PDF - Accepted Version
Restricted to Registered users only


DOI: 10.1109/ICDMW.2010.49


Handling changes over time in supervised learning (concept drift) lately has received a great deal of attention, a number of adaptive learning strategies have been developed. Most of them make an optimistic assumption that the new labels become available immediately. In real sequential classification tasks it is often unrealistic due to task specific delayed labeling or associated labeling costs. We address the problem of change detectability, given, that the new labels are not available. In this analytical study we look at the space of changes from probabilistic perspective to analyze, what changes are detectable without seeing the labels and what are not. We conduct a range of experiments with real life data with simulated and natural changes to explore this detectability issue. We propose a computationally friendly detection technique, which monitors a stream of classifier outputs. We demonstrate analytically and experimentally, what types of changes are possible to detect when the labels for the new data are not available.

Item Type:Conference or Workshop Item (Paper)
Subjects:Generalities > Computer Science and Informatics > Artificial Intelligence
Group:Faculty of Science and Technology
ID Code:17635
Deposited By:Dr Indre Zliobaite LEFT
Deposited On:21 Apr 2011 14:56
Last Modified:10 Sep 2014 15:51

Available Versions of this Item


Downloads per month over past year

More statistics for this item...
Repository Staff Only -