Third Workshop on
ROC Analysis in ML
Pittsburgh, USA, 29 June, 2006
held within ICML’2006, the 23rd International Conference on Machine Learning
Receiver Operating Characteristic Analysis (ROC Analysis) is related in a direct and natural way to cost/benefit analysis of diagnostic decision making. Widely used in medicine for many decades, it has been introduced relatively recently in machine learning. In this context, ROC analysis provides tools to select possibly optimal models and to discard suboptimal ones independently from (and prior to specifying) the cost context or the class distribution. Furthermore, the Area Under the ROC Curve (AUC) has been shown to be a better evaluation measure than accuracy in contexts with variable misclassification costs and/or imbalanced datasets. AUC is also the standard measure when using classifiers to rank examples, and, hence, is used in applications where ranking is crucial, such as campaign design, model combination, collaboration strategies, and co-learning.
Nevertheless, there are many open questions and some limitations that hamper a broader use and applicability of ROC analysis. Its use in data mining and machine learning is still below its full potential. An important limitation of ROC analysis, despite some recent progress, is its possible but difficult extension for more than two classes.
This workshop follows up a first workshop (ROCAI'04) held within ECAI-2004 and a second workshop (ROCML'05) held within ICML-2005. This third workshop is intended to investigate on the hot topics identified during the two previous workshops (e.g. multiclass extension, statistical analysis, alternative approaches), on the one hand, and to encourage cross-fertilisation with ROC practitioners in medicine, on the other hand, thanks to an invited medical expert.
Dr. Darrin C. Edwards, Department of Radiology, University of Chicago
will present the state of the art of ROC analysis in
radiology. Moreover a three-class medical dataset is available to
support exchanges between medical experts and participants. We
strongly encourage submissions dealing with the provided medical
dataset. The three-class
medical dataset will be available on this webpage soon. Please send an
email at firstname.lastname@example.org
if you want to join the mailing list where updates will be sent.
We encourage submissions on hot topics raised during previous
editions, e.g. ROC analysis software repository, multiclass extension,
statistical analysis. To promote a
workshop atmosphere the program committee will select a few topics and
relevant papers. The program and accepted papers will be published on
the workshop website before the call for participation where we will
encourage people to read them in order to focus on the discussion
during the workshop. The authors of accepted papers will be asked to
prepare the discussion rather than to detail their paper, by
summarising their contributions, on the one hand, and by comparing them
to the other points of view presented in their session, on the other
The attendance to the workshop is open to all. To register the workshop see ICML'06 registration procedure. It is possible to register for the workshop only (select fee waived option at
Organizing Committee Submission
Guidelines Potential participants are invited to
submit papers according to one of the following formats: Authors should submit their
papers electronically (PDF or PS format) to the contact person (email@example.com)
submit papers using the final camera-ready ICML 2006 conference paper style,
including author names. Some
(introduction to ROC analysis, software, etc.)
Workshop Organizing Committee
Potential participants are invited to
submit papers according to one of the following formats:
Authors should submit their papers electronically (PDF or PS format) to the contact person (firstname.lastname@example.org) It is recommended to submit papers using the final camera-ready ICML 2006 conference paper style, including author names.
Some relevants links (introduction to ROC analysis, software, etc.)