[visionlist] New task at ImageCLEF 2008: retrieval of wikipedia images

Theodora Tsikrika Theodora.Tsikrika at cwi.nl
Sun Mar 16 21:09:24 GMT 2008


       *** Call for participation: wikipediaMM image retrieval 2008 ***

In 2008, ImageCLEF introduces a new ad hoc image retrieval task that
provides a testbed for the system-oriented evaluation of visual
information retrieval from a collection of Wikipedia images. The aim is to
investigate multimodal retrieval approaches in the context of a larger
scale and heterogeneous collection of images (similar to those encountered
on the Web) searched by users with diverse information needs.

This task is called wikipediaMM and will use the image collection created
and employed by the INEX Multimedia (MM) Track (2006-2007) that contains
approximately 150,000 images covering diverse topics of interest. These
images are associated with unstructured and noisy textual annotations in

The main aim is to investigate:
  * how well do the retrieval approaches cope with larger scale image
  * how well do the retrieval approaches cope with noisy and unstructured
textual annotations?
  * how well do the content-based retrieval approaches cope with images
that cover diverse topics and are of varying quality?
  * how well can systems exploit and combine different modalities given a
user's multimedia information need? Can they outperform monomodal
approaches like query-by-text, query-by-concept or query-by-image?

If you are interested in participating in this task, read on and/or go to
the ImageCLEF web site and register.



The wikipediaMM task adopts the user model followed in INEX, whereby the
participants in the various tracks create (some of) the topics and perform
the relevance assessments themselves.

Therefore, participation in ImageCLEF's wikipediaMM task requires that
each participating group:
    * creates topics
    * performs the relevance assessments on the created topics

Note that only those who participate in the topic development and
assessment process will be granted access to the relevance assessments.

The schedule can be found here:

    * 20.2.2008: registration opens for all CLEF tasks
    * 15.3.2008: data release
    * 17.3.2008: instructions and formatting criteria for candidate
topics/queries provided to participants
    * 10.4.2008: submission deadline for candidate topics
    * 16.4.2008: topic release
    * 30.5.2008: submission of runs
    * 4.6.2008: distribution of merged results to participants for
relevance assessments
    * 7.7.2008: submission deadline for relevance assessments
    * 15.7.2008: release of results
    * 15.8.1008: submission of working notes papers
    * 17.-19.9.2008: CLEF workshop in Aarhus, Denmark

If you have any questions, do not hesitate to contact us.


* Theodora Tsikrika, Database Architectures and Information Access Group,
CWI, Amsterdam, The Netherlands, Theodora.Tsikrika at cwi.nl
* Jana Kludas, Computer Vision and Multimedia Laboratory, University of
Geneva, Switzerland, Jana.Kludas at cui.unige.ch

CWI, room C1.17          Centre for Mathematics and Computer Science
Kruislaan 413                        Email: Theodora.Tsikrika at cwi.nl
1098 SJ Amsterdam                    tel:       +31-(0)20-5924319
The Netherlands                      fax:       +31-(0)20-5924312

More information about the visionlist mailing list