[visionlist] Call-for-Participation: 2nd Fusion Task @ ImageCLEF 2023 (Machine Learning System Fusion)

[Bogdan Ionescu]

[Apologies for multiple postings]

ImageCLEFfusion (2nd edition)
Registration: https://www.imageclef.org/2023/fusion
Run submission: May 10, 2023
Working notes submission: June 5, 2023
CLEF 2023 conference: September 18-21, Thessaloniki, Greece


*** CALL FOR PARTICIPATION ***
While deep neural networks have proven their predictive power in many
tasks, there are still several domains where a single deep learning
network is not enough for attaining high precision, e.g., prediction
of subjective concepts such as violence, memorability, etc.

Late fusion, also called ensembling or decision-level fusion,
represents one of the approaches that researchers employ to increase
the performance of single-system approaches. It consists of using a
series of weaker learner methods called inducers, whose prediction
outputs are combined in the final step, via a fusion mechanism to
create a new and improved super predictor. These systems have a long
history and are shown to be particularly useful in scenarios where the
performance of single-system approaches is not considered
satisfactory.

The task challenges participants to develop and benchmark late fusion
schemes. This task would allow to explore various aspects of late
fusion mechanisms, such as the performance of different fusion
methods, the methods for selecting inducers from a larger set, the
exploitation of positive and negative correlations between inducers,
and so on.


*** TASK ***
The participants will receive a data set of real inducers and are
expected to provide a fusion mechanism that would allow to combine
them into a super-system yielding superior performance compared to the
highest performing individual system. The provided inducers were
developed to solve three real tasks:
(i) prediction of visual interestingness (int --- regression task),
(ii) diversification of image search results (div --- retrieval task),
(iii) medical image captioning (cap --- multi-class labeling task).


*** DATA SET ***
ImageCLEFfusion-int. The data for this task is extracted and
corresponds to the Interestingness10k dataset. We will provide output
data from 33 inducers, while 1,826 samples will be used for the
development set, and 609 samples will be used for the testing set.
ImageCLEFfusion-div. The data for this task is extracted and
corresponds to the Retrieving Diverse Social Images Task dataset. We
will provide outputs data from 117 inducers, while 104 queries will be
used for the development set, and 35 samples will be used for the
testing set.
ImageCLEFfusion-cap. The data for this task is extracted from the
ImageCLEFmedical Caption task. We will provide output data from 85
inducers, while 5,700 images will be used for the development set, and
1900 images will be used for the testing set.


*** METRICS ***
Evaluation will be performed using the metrics specific to each
dataset we use, e.g., MAP at 10, F1 at 20, ClusterRecall at 20, accuracy.


*** IMPORTANT DATES ***
- Run submission: May 10, 2023
- Working notes submission: June 5, 2023
- CLEF 2023 conference: September 18-21, Thessaloniki, Greece
(https://clef2023.clef-initiative.eu/)


*** OVERALL COORDINATION ***
Liviu-Daniel Stefan, Politehnica University of Bucharest, Romania
Mihai Gabriel Constantin, Politehnica University of Bucharest, Romania
Mihai Dogariu, Politehnica University of Bucharest, Romania
Bogdan Ionescu, Politehnica University of Bucharest, Romania


*** ACKNOWLEDGEMENT ***
The task is supported under the H2020 AI4Media “A European Excellence
Centre for Media, Society and Democracy” project, contract #951911
https://www.ai4media.eu/.


On behalf of the Organizers,

Bogdan Ionescu
https://www.AIMultimediaLab.ro/