[visionlist] Call-for-Participation: Aware Task @ ImageCLEF 2023 (Unveiling Real-Life Effects of Online Photo Sharing)

[Bogdan Ionescu]

[Apologies for multiple postings]

ImageCLEFaware (3rd edition)
Registration: https://www.imageclef.org/2023/aware
Run submission: May 10, 2023
Working notes submission: June 5, 2023
CLEF 2023 conference: September 18-21, Thessaloniki, Greece


*** CALL FOR PARTICIPATION ***
Images constitute a large part of the content shared on social
networks. Their disclosure is often related to a particular context
and users are often unaware of the fact that, depending on their
privacy status, images can be accessible to third parties and be used
for purposes which were initially unforeseen. For instance, it is
common practice for employers to search information about their future
employees online. Another example of usage is that of automatic credit
scoring based on online data. Most existing approaches which propose
feedback about shared data focus on inferring user characteristics and
their practical utility is rather limited.

We hypothesize that user feedback would be more efficient if conveyed
through the real-life effects of data sharing.

The objective of the task is to automatically score user photographic
profiles in a series of situations with strong impact on her/his life.
Four such situations were modeled this year and refer to searching
for: (i) a bank loan, (ii) an accommodation, (iii) a job as
waitress/waiter, and (iv) a job in IT. The inclusion of several
situations is interesting in order to make it clear to the end-users
of the system that the same image will be interpreted differently
depending on the context.

The final objective of the task is to encourage the development of
efficient user feedback, such as the YDSYO Android app
https://ydsyo.app/.


*** TASK ***
Given an annotated training dataset, participants will propose machine
learning techniques which provide a ranking of test user profiles in
each situation which is as close as possible to a human ranking of the
test profiles.


*** DATA SET ***
This is the third edition of the task. A data set of more than 1,000
user profiles with 100 photos per profile was created and annotated
with an appeal score for a series of real-life situations via
crowdsourcing. Participants to the experiment were asked to provide a
global rating of each profile in each situation modeled using a
7-points Likert scale ranging from strongly unappealing to strongly
appealing. An averaged and normalized appeal score will be used to
create a ground truth composed of ranked users in each modeled
situation. User profiles are created by repurposing a subset of the
YFCC100M dataset.


*** METRICS ***
Participants to the task will provide an automatically ranking of user
ratings for each situation which will be compared to a ground truth
rating obtained by crowdsourcing. The correlation between the two
ranked list will be measured using Pearson's correlation coefficient.
The final score of each participating team will be obtained by
averaging correlations obtained for individual situations.


*** IMPORTANT DATES ***
- Run submission: May 10, 2023
- Working notes submission: June 5, 2023
- CLEF 2023 conference: September 18-21, Thessaloniki, Greece
(https://clef2023.clef-initiative.eu/)


*** OVERALL COORDINATION ***
Jérôme Deshayes-Chossart, CEA LIST, France
Adrian Popescu, CEA LIST, France
Bogdan Ionescu, Politehnica University of Bucharest, Romania


*** ACKNOWLEDGEMENT ***
The task is supported under the H2020 AI4Media “A European
Excellence Centre for Media, Society and Democracy† project,
contract #951911 https://www.ai4media.eu/.


On behalf of the Organizers,

Bogdan Ionescu
https://www.AIMultimediaLab.ro/