[visionlist] CLEF2021: 1st Call-for-Labs-Participation

[Bogdan Ionescu]

[Apologies for multiple postings]

CLEF 2021 Conference and Labs on the Evaluation Forum
Information Access Evaluation meets Multilinguality, Multimodality and
Visualization
21 - 24 September 2021, Bucharest - Romania


*** CALL-FOR-LAB-PARTICIPATION ***
Conference website: http://clef2021.clef-initiative.eu/

CLEF 2021 is the 12th CLEF conference continuing the popular CLEF
campaigns which have run since 2000 contributing to the systematic
evaluation of information access systems.

It consists of a peer-reviewed conference (see the separate call for
papers) and a set of 12 Labs designed to test different aspects of
multilingual and multimedia IR systems:
1. ARQMath: Answer Retrieval for Questions on Math
2. BioASQ: Large-scale biomedical semantic indexing and question answering
3. CheckThat!: Lab on Detecting Check-Worthy Claims, Previously
Fact-Checked Claims, and Fake News
4. ChEMU: Cheminformatics Elsevier Melbourne University lab
5. CLEF eHealth: medical content retrieval to support decision making
6. eRisk: Early Risk prediction on the Internet
7. ImageCLEF: Multimedia Retrieval Challenge in CLEF
8. LifeCLEF: Multimedia Life Species Identification
9. LiLAS: Living Labs for Academic Search
10. PAN: Digital Text Forensics and Stylometry
11. SimpleText-202: (Re)Telling right scientific stories to
non-specialists via text simplification
12. Touché: Argument Retrieval


*** IMPORTANT DATES ***
Lab Registration Opens: 16 November 2020
Registration closes: 30 April 2021
End Evaluation Cycle: 7 May 2021
Working Notes papers due: 28 May 2021
Camera Ready Copy: 2 July 2021


*** PARTICIPATION ***
Lab participants must register for the Labs via the CLEF website:
http://clef2021-labs-registration.dei.unipd.it/


*** LABS ***

1. ARQMath: Answer Retrieval for Questions on Math
ARQMath aims to advance math-aware search and the semantic analysis of
mathematical notation and texts.
*Task 1 - Answer Retrieval: Given a math question post, return
relevant answer posts.
*Task 2 - Formula Search: Given a formula in a math question post,
return relevant formulas from both question and answer posts.

Lab Coordination: Richard Zanibbi (Rochester Institute of Technology),
Douglas Oard (University of Maryland), Anurag Agarwal (Rochester
Institute of Technology), and Behrooz Mansouri (Rochester Institute of
Technology)
Lab Website: https://www.cs.rit.edu/~dprl/ARQMath
Twitter: @ARQMath1


2. BioASQ: Large-scale biomedical semantic indexing and question answering
The aim of the BioASQ Lab is to push the research frontier towards
systems that use the diverse and voluminous information available
online to respond directly to the information needs of biomedical
scientists.
*Task 1 - Task A: Large-Scale Online Biomedical Semantic Indexing.
*Task 2 - Task B: Biomedical Semantic Question Answering.
*Task 3 - Task MESINESP: Medical Semantic Indexing In Spanish.
*Task 4 - Task Synergy: Question Answering for COVID-19.

Lab Coordination: Anastasia Krithara (National Center for Scientific
Research "Demokritos"), Anastasios Nentidis (National Center for
Scientific Research "Demokritos" & Aristotle University of
Thessaloniki), George Paliouras (National Center for Scientific
Research "Demokritos"), Martin Krallinger (Barcelona Supercomputing
Center), and Marta Villegas (Barcelona Supercomputing Center)
Lab Website: http://www.bioasq.org/workshop2021
Twitter: @BioASQ


3. CheckThat!: Lab on Detecting Check-Worthy Claims, Previously
Fact-Checked Claims, and Fake News
The CheckThat! Lab aims at fighting misinformation and disinformation
in social media, in political debates, and in the news, with focus on
three tasks (in English, Arabic, German and Spanish): check-worthiness
estimation, detecting previously fact-checked claims, and fake news
detection.
*Task 1 - Check-worthiness estimation: given a piece of text, detect
whether it is worth fact-checking.
*Task 2 - Detecting previously fact-checked claims: given a
check-worthy claim in the form of a tweet, and a set of
previously-checked claims, determine whether the claim has been
previously fact-checked.
*Task 3 - Fake new detection: this task has two subtasks: multi-class
fake news detection, and domain identification of fake news articles.

Lab Coordination: Preslav Nakov, Giovanni da San Martino, Tamer
Elsayed, Alberto Barron-Cedeno,Ruben Miguez, Shaden Shaar, Firoj Alam,
Fatima Haouari, Maram Hasanain, Nikolay Babulkov, Alex Nikolov, Thomas
Mandl, Julia Maria Struß, Gautam Kishore Shahi, and Sandip Modha
Lab Website: https://sites.google.com/view/clef2021-checkthat


4. ChEMU: Cheminformatics Elsevier Melbourne University lab
ChEMU (Cheminformatics Elsevier Melbourne University) lab aims to
advance the state-of-the-arts in information extraction over chemical
patents, and provide the opportunity to address the fundamental NLP
task - anaphora resolution in the context of chemical patents.
*Task 1 - Chemical Reaction Reference Resolution: given a description
of a chemical reaction, the task requires identification of other
chemical reactions or general conditions that it refers to.
*Task 2 - Anaphora Resolution: this task focuses on expression-level
references, and requires identification of reference resolution
between expressions in chemical patents.

Lab Coordination: Karin Verspoor (The University of Melbourne), Tim
Baldwin (The University of Melbourne), Trevor Cohn (The University of
Melbourne), Saber Akhondi (Elsevier), Jiayuan He (The University of
Melbourne), Christian Druckenbrodt (Elsevier), Camilo Thorne
(Elsevier), Biaoyan Fang (The University of Melbourne), Hiyori
Yoshikawa (The University of Melbourne), and Zenan Zhai (The
University of Melbourne)
Lab Website: http://chemu.eng.unimelb.edu.au/


5. CLEF eHealth: medical content retrieval to support decision making
This lab aims to support the development of techniques to aid
laypeople, clinicians and policy-makers in easily retrieving and
making sense of medical content to support their decision making.
*Task 1 - Information Extraction from Noisy Text: participants will
identify and classify Named Entities in written ultrasonography
reports, containing misspellings and inconsistencies, from a major
public hospital in Argentina.
*Task 2 - Consumer Health Search: participants must retrieve web pages
that fulfil a given patient’s personalised information need. This
needs to fulfil the following criteria: information credibility,
quality, and suitability.

Lab Coordination: Laura Alonso Alemany (Univ. Nacional de Córdoba),
Viviana Cotik (Univ. de Buenos Aires), Lorraine Goeuriot (Univ.
J.Fourier), Liadh Kelly (Maynooth University), Gabriella Pasi
(University of Milano-Bicocca), and Hanna Suominen (Australian
National Univ.)
Lab Website: https://clefehealth.imag.fr/
Twitter: @clefhealth


6. eRisk: Early Risk prediction on the Internet
The main purpose of eRisk is to explore issues of evaluation
methodology, effectiveness metrics and other processes related to
early risk detection.
*Task 1 - Early detection of pathological gambling: this task will
pursue early detection models for emitting alerts on users starting to
develop this mental health problem.
*Task 2 - Early detection of self-harm: this task will pursue early
detection models for emitting alerts on users starting to develop this
mental health problem.
*Task 3 - Depression-level estimation: in this task, participants are
expected to develop models to automatically fill a standard depression
questionnaire based on the user's writings.

Lab Coordination: Javier Parapar (Universidade da Coruña), Patricia
Martín-Rodilla (Universidade da Coruña), David E. Losada (Universidade
de Santiago de Compostela), and Fabio Crestani (University of Lugano)
Lab Website: https://erisk.irlab.org/
Twitter: @earlyrisk


7. ImageCLEF: Multimedia Retrieval Challenge in CLEF
ImageCLEF 2021, the Multimedia Retrieval Challenge in CLEF, is set to
promote the evaluation of technologies for annotation, indexing,
classification and retrieval of multi-modal data, with the objective
of providing information access to large collections of images in
various usage scenarios and domains.
*Task 1 - ImageCLEFmedical: participants will be requested to develop
solutions for automatically identifying individual components from
which captions are composed in Radiology Objects in COntext images and
to combine natural language processing and computer vision for
answering a medical question based on a visual image content.
*Task 2 - ImageCLEFcoral: requires participants to automatically
segment and label a collection of coral images that can be used in
combination to create 3-dimensional models of an underwater
environment.
*Task 3 - ImageCLEFdrawnUI: requires participants to detect
rectangular bounding boxes corresponding to website User Interface
elements from hand-drawn and real-world images.
*Task 4 - ImageCLEFaware: participants are required to provide
automatic rankings of photographic user profiles in a series of
real-life situations such as searching for a job, accommodation,
insurance, bank loans, etc.

Lab Coordination: Bogdan Ionescu (University Politehnica of
Bucharest), Henning Müller (University of Applied Sciences Western
Switzerland), and Renaud Péteri (University of La Rochelle)
Lab Website: https://www.imageclef.org/2021
Twitter: @imageclef


8. LifeCLEF: Multimedia Life Species Identification
LifeCLEF lab aims at boosting research on the identification and
prediction of living organisms in order to solve the taxonomic gap and
improve our knowledge of biodiversity.
*Task 1 - PlantCLEF: image-based plant species identification from
herbarium sheets.
*Task 2 - BirdCLEF: bird species identification from bird calls and
songs in audio soundscapes.
*Task 3 - GeoLifeCLEF: location-based species prediction based on
environmental and occurrence data.
*Task 4 - SnakeCLEF: image-based and location-based snake identification.

Lab Coordination: Alexis Joly (Inria, LIRMM), Henning Müller (HES-SO),
and Hervé Goëau (CIRAD UMR AMAP)
Lab Website: https://www.imageclef.org/LifeCLEF2021


9. LiLAS: Living Labs for Academic Search
In the LiLAS lab, we would like to bring together IR researchers
interested in the online evaluation of academic search systems. The
goal is to foster knowledge on improving the search for academic
resources like literature, research data, and the interlinking between
these resources.
*Task 1 - Ad-hoc Search Ranking: the participants get access to the
LIVIVO corpus that consists of about 80 million documents from more
than 50 data sources in multiple languages from the Life Sciences.
*Task 2 - Research Data Recommendations: the goal is to provide
recommendations of research data that are relevant and related to the
publication the user is currently viewing. The data for this task is
taken from the academic search system GESIS Search from the Social
Sciences.

Lab Coordination: Philipp Schaer (TH Köln - University of Applied
Sciences), Johann Schaible (GESIS - Leibniz Institute for the Social
Sciences), and Leyla Jael Garcia Castro (ZB MED - Information Centre
for Life Sciences)
Lab Website: https://clef-lilas.github.io


10. PAN: Digital Text Forensics and Stylometry
PAN is a series of scientific events and shared tasks on digital text
forensics and stylometry, studying how to quantify writing style and
improve authorship technology.
*Task 1 - Cross-domain Authorship Verification: given two texts,
determine if they are written by the same author.
*Task 2 - Multi-Author Writing Style Analysis: given a text written by
two or more authors, find all positions of writing style change.
*Task 3 - Profiling Haters on Twitter: given a user's Twitter
timeline, determine if they spread hate speech.

Lab Coordination: Martin Potthast (Leipzig University), Paolo Rosso
(Universitat Politècnica de València), Efstathios Stamatatos
(University of the Aegean), and Benno Stein (Bauhaus-Universität
Weimar)
Lab Website: https://pan.webis.de/
Twitter: @webis_de


11. SimpleText-202: (Re)Telling right scientific stories to
non-specialists via text simplification
SimpleText aims at contributing to making science more open &
accessible via automatic generation of simplified summaries of
scientific documents.
*Pilot Task 1 - Searching for background information: given a
scientific text, provide background information from an external
source to help a user to understand it (definitions, context, etc.).
*Pilot Task 2 - Text Simplification: given a short scientific text
(e.g., abstract), generate its simplified version.

Lab Coordination: Liana Ermakova, Patrice Bellot, Pavel Braslavski,
Jaap Kamps, Josiane Mothe, Diana Nurbakova, and Eric San-Juan
Lab Website: https://www.irit.fr/simpleText/
Twitter: @SimpletextW


12. Touché: Argument Retrieval
Touché has been designed with the goal to establish a collaborative
platform for researchers in the field of argument retrieval and to
provide tools for developing and evaluating argument retrieval
approaches.
*Task 1 - Conversational Argument Retrieval: given a query on a
controversial topic, the task is to retrieve relevant arguments from a
focused crawl of online debate portals.
*Task 2 - Comparative Argument Retrieval: given a comparative
question, the task is to retrieve and rank documents from the
ClueWeb12 that help to answer the comparative question with arguments
and help to make an informed decision.

Lab Coordination: Alexander Bondarenko (Martin-Luther-Universität
Halle-Wittenberg), Lukas Gienapp (Leipzig University), Maik Fröbe
(Martin-Luther-Universität Halle-Wittenberg), Meriem Beloucif
(Universität Hamburg), Yamen Ajjour (Martin-Luther-Universität
Halle-Wittenberg), Alexander Panchenko (Skolkovo Institute of Science
and Technology), Chris Biemann (Universität Hamburg), Benno Stein
(Bauhaus-Universität Weimar), Henning Wachsmuth (Paderborn
University), Martin Potthast (Leipzig University), and Matthias Hagen
(Martin-Luther-Universität Halle-Wittenberg)
Lab Website: https://touche.webis.de
Twitter: @webis_de


*** ORGANIZERS ***

* General Chairs *
Bogdan Ionescu, University "Politehnica" of Bucharest, Romania
K. Selcuk Candan, Arizona State University, USA

* Program Chairs *
Henning Müller, University of Applied Sciences Western Switzerland, Switzerland
Lorraine Goeuriot, Université Grenoble Alpes, France
Birger Larsen, Aalborg University Copenhagen, Denmark

* Evaluation Lab Chairs *
Alexis Joly, INRIA Sophia-Antipolis, France
Maria Maistro, University of Copenhagen, Denmark
Florina Piroi, Vienna University of Technology, Austria

* Publicity Chairs *
Liviu Stefan Daniel, University "Politehnica" of Bucharest, Romania
Mihai Dogariu, University "Politehnica" of Bucharest, Romania

* Outreach Program Chairs *
Yu-Gang Jiang, Fudan University, China - Asian Liaison
Hugo Jair Escalante, Instituto Nacional de Astrofisica, Optica y
Electronica, Mexico - Central American Liaison
Fabio A. Gonzalez, National University of Colombia, Colombia - South
American Liaison
Ben Herbst, Praelexis, South Africa - African Liaison
Abdulmotaleb El Saddik, University of Ottawa, Canada - North American Liaison

* Industry & Sponsorship Chairs *
Seila Abdulamit, Vodafone, Romania
Mihai-Gabriel Constantin, University "Politehnica" of Bucharest, Romania
Bogdan Boteanu, University "Politehnica" of Bucharest, Romania

* Website & Social Media Chairs *
Denisa Ionascu, University "Politehnica" of Bucharest, Romania

* Finance Chair *
Ion Marghescu, University "Politehnica" of Bucharest, Romania

* Proceedings Chairs *
Guglielmo Faggioli, University of Padua, Italy
Nicola Ferro, University of Padua, Italy


On behalf of the organizers,

Bogdan Ionescu
https://bionescu.aimultimedialab.ro/