[visionlist] [Challenge] The First Perception Test Challenge @ICCV23

[Viorica Patraucean]

The First Perception Test Challenge ICCV23 Workshop

Website: <https://sites.google.com/view/iccv23trickyworkshop>
https://ptchallenge-workshop.github.io/

We are happy to announce that the first Perception Test challenge will be
held in conjunction with ICCV2023.

*****************************

OVERVIEW

*****************************

Making progress towards building perception systems that can achieve human
level scene understanding requires robust and comprehensive evaluation
benchmarks to reveal the strengths and weaknesses of these models and guide
research. We organise this challenge-workshop to benchmark multimodal
perception models using the Perception Test (blog
https://www.deepmind.com/blog/measuring-perception-in-ai-models, github
https://github.com/deepmind/perception_test) -- a diagnostic benchmark
created by DeepMind to probe the abilities of multimodal models across
video, audio, and text modalities, in four skill areas (Memory,
Abstraction, Physics, Semantics), four types of reasoning (descriptive,
explanatory, predictive, counterfactual), and six computational tasks
(multiple-choice video-QA, grounded video-QA, object tracking, point
tracking, action localisation, sound localisation). The focus is on
transfer and generalisation capabilities, so zero-shot/few-shot evaluation
is prioritised. A limited training set is available if models require
fine-tuning.

Invited speakers: Derek Hoiem (University of Illinois Urbana-Champaign),
Rohit Girdhar (Facebook AI Research).

Workshop organisers: Viorica Patraucean (Google DeepMind), Joao Carreira
(Google DeepMind), Dima Damen (University of Bristol), Andrew Zisserman
(University of Oxford)


*****************************

CHALLENGE TRACKS

*****************************

Multiple-choice videoQA
https://eval.ai/web/challenges/challenge-page/2091/overview

Grounded videoQA https://eval.ai/web/challenges/challenge-page/2110/overview


Object tracking https://eval.ai/web/challenges/challenge-page/2094/overview

Point tracking https://eval.ai/web/challenges/challenge-page/2108/overview

Temporal action localisation
https://eval.ai/web/challenges/challenge-page/2101/overview

Temporal sound localisation
https://eval.ai/web/challenges/challenge-page/2109/overview



*****************************

IMPORTANT DATES:

*****************************

Evaluation server goes live with validation split: June-July 2023

Held-out test set released: August 1st, 2023

Deadline for submissions: September 16th, 2023

Workshop: October 3rd 2023, morning session


*****************************

PRIZES

*****************************

Prizes up to EUR 15K will be offered to top competitors across all
challenge tracks.


*****************************

CONTACT

*****************************

E-mail:
<http://visionscience.com/mailman/listinfo/visionlist_visionscience.com>
perception-test at google.com, viorica at google.com
Website: <https://sites.google.com/view/iccv23trickyworkshop>
https://ptchallenge-workshop.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20230822/cb6b247e/attachment.html>