Menu:

Latest news:

Jan, 2010:
Webpage updated.

Links:

Contests and Competitions

Contests and competitions play an important role in measuring the state of the art in pattern recognition applications. The provide object performance measures and show which methods can be tuned to what extend to obtain really good results which allows valuable insights into which direction of research is promising to be followed in the future.

NEW

Pascal Challenge on Large Scale Hierarchical Text classification Link

Regular competitions

ImageCLEF
An evaluation campaign that organizes evaluations for content-based and text-based retrieval of image retrieval systems on various tasks annually since 2003.

TrecVID Video Retrieval Evaluation
Video Retrieval evaluation running annually as a subtrack in TREC which uses multi-model information (speech recognition, videos, textual annotation) to retrieve video sequences from different type of media.

Cross Language Evaluation Forum
Evaluation Campagin to evaluate cross language and language independet information retrieval methods with an annual schedule.

KDD competition
Data mining competition with different tasks. Run annually in combination with the KDD (Knowledge discovery in databases) conference.

One Time Competitions and former regular competitions (latest first)

MLSP 2008 Machine Learning Competition: Stock Market Prediction. The competition this year is to design a machine learning method to predict the stock market. Each entrant will start with $100,000 fictitious dollars. The machine learning algorithm will need to select stocks to purchase/sell/cover/short for a test set that covers 6 months worth of actual stock prices.

SHREC 08 - Shape Retrieval Contest 2008. A shape retrieval contest including five different tracks!

ICAPS Competitions
The International Planning Competition organised in conjunction with the International Conference on Automated Planning and Scheduling offers a machine learning track in 2007.

NICE.I: Noisy Iris Segmentation contest
The NICE.I contest is a completely free-of-charge iris segmentation and noise detection for biometric purposes contest that is being held by the SOCIA Lab. of the University of Beira Interior (Portugal)

ImagEval
An evaluation campaign for content-based access methods to images which ran an evaluation in 2006/2007.

GREC 2007 Arc Segmentation Contest
Aompetition regarding the detection of arcs in digital images held at the Graphics Recognition Workshop GREC

PASCAL Object Classes Challenge
An evaluation campaign for object recognition and detection sponsored by the PASCAL network of excellence which ran challenges in 2005, 2005 and 2007.

CIVR 2007 evaluation showcases
A live image retrieval demosession hosted at CIVR

Coin Image Classification Evaluation
Coin classification challenge sponsored by the MUSCLE network of excellence.

Data mining cup
A competition for students (undergrad) run annually by prudsys and Chemnitz University of Technology in Chemnitz, Germany.

NetFlix
A data mining competition which is run by netflix. Objective is to predict customer preferences for videos.

BCI Competition III
The goal of the "BCI Competition III" is to validate signal processing and classification methods for Brain-Computer Interfaces (BCIs).

ICDAR 2005 Robust Reading Competitions
Competitions related to locating and reading text in natural scenes. Competitions are: locating text, character recognition, word recognition, and overall reading problem.

Omphalos - ICGI 2004 Context-Free Grammar Learning Competition
Omphalos is a context-free language learning competition that forms part of ICGI-2004. The task is to infer a model of a context-free language from unstructured examples (both positive and negative) and to use that model to label a set of test sentences as being either in or out of the language.

ICML Physiological Data Modeling Contest
Physiological data offers many challenges to the machine learning community including dealing with large amounts of data, sequential data, issues of sensor fusion, and a rich domain complete with noise, hidden variables, and significant effects of context. The Physiological Data Modeling Contest will use data collected using wearable BodyMedia body monitors.

CEC 2004 Computer Controlled Car Racing
The aim of this contest is to find the best race controller for a remote control car. The controller is given live video feed from an overhead or car-mounted camera, and after processing each frame can then send signals to the car in order to achieve maximum progress around the track, relative to the opponent car. Ramming and blocking tactics are encouraged!

Learning DFA from noisy Samples
The challenge is as follows: given a training set of binary sequences, each with a binary class label, learn a predictor that will classify unlabelled sequences in the test set. The training and test sets consist of random binary sequences labelled by a randomly constructed Deterministic Finite Automata (DFA). Note that your predictor can take any form you want: a neural net, an expression tree, etc., it does not have to be a DFA. The training set class labels are corrupted with noise, which makes the learning problem much harder.

ICPR 2004 Face Verification Contest
Contest on the BANCA dataset - financial prize for winner to be presented at the ICPR 2004 conference dinner

2004 International Joint Conference on Neural Networks TIME SERIES PREDICTION COMPETITION
The goal of this competition is to provide a new benchmark for the problem of time series predictionand to compare the different methods or models that can be used for the prediction.The proposed time series is the CATS benchmark (for Competition on Artificial Time Series).

CEC 2003 Remote Control Car Racing
The aim of this contest is to find the best race controller for a remote control car. The controller is given live video feed from an overhead or car-mounted camera, and after processing each frame can then send signals to the car in order to achieve maximum progress around the track, relative to the opponent car. Ramming and blocking tactics are allowed, and if effective are applauded! Interesting pattern recognition challenges in real-time object tracking.

NIPS 2003 Feature Selection Challenge
Five two-class datasets: the aim of the challenge in feature selection is to find feature selection algorithms that significantly outperform methods using all features, on ALL five benchmark datasets.

First International Signature Vecrification Competition (SVC 2004)
Held in conjunction with the First International Conference on Biometric Authentication (ICBA 2004). (Note that the date quoted here is for registration - code submission deadline is 31 December 2003)

Fingerprint Vecrification Competition (FVC 2004)
Held in conjunction with the First International Conference on Biometric Authentication (ICBA 2004). (Note that the date quoted here is for registration - code submission deadline is 30 November 2003)

ICDAR 2003 Competitions
Various competitions, including reading text in natural scenes, page segmentation and cursive script recognition (i.e. reading hand-writing).

Gowachin: Learning DFA in Noise
The successor to Abbadingo One. A continuosly running contest to learn DFA from samples of data. You select the approximate size of DFA, the number of sample strings and the noise level (which can disrupt either the string symbols or the class labels, or both.