[visionlist] [meetings][CfP] ECCV 2020 Workshop - Beyond mAP: Reassessing the Evaluation of Object Detectors

[David Hall]

 ECCV 2020 Workshop - Beyond mAP: Reassessing the Evaluation of Object Detectors

Dear Researchers,

Not long now before papers are due for our ECCV workshop.

https://nikosuenderhauf.github.io/roboticvisionchallenges/eccv2020

This workshop assesses current evaluation procedures for object detection, highlights their shortcomings and opens discussion for possible improvements.

Below are some updates on the workshop for if you are interested in submitting to the workshop, competing in the attached probabilistic object detection (PrOD) challenge, or just attending the workshop itself.

Workshop Interactive Sessions + Invited Speakers:

Submit questions for discussion by our panel/community through our Slido page
https://app.sli.do/event/v4q9zeul

Session 1 - 00:00 - 02:00 UTC+1 August 28th

  *   Assistant Prof. Walter Scheirer (University of Notre Dame)
  *   Dr Larry Zitnick (Facebook AI Research)
  *   Dr David Hall (Queensland University of Technology)

Session 2 - 08:00 - 10:00 UTC+1 August 28th

  *   Assistant Prof. Emre Akbas (Middle East Technical University)
  *   Dr David Hall (Queensland University of Technology)

Important Dates:

  *   05 August 2020 Final PrOD submissions to the Evaluation server via CodaLab
  *   07 August 2020 Paper Submissions due (Workshop and Challenge) via CMT<https://cmt3.research.microsoft.com/BMREOD2020>
  *   12 August 2020 Winner Announcements and Workshop Invitations
  *   15 August 2020 Videos and photo-ready papers for workshop due
  *   28 August 2020 ECCV Workshop Interactive sessions

Paper Topics of Interest (guidelines):

  *   New evaluation measures/metrics for object detection
  *   New evaluation/visualization tools to analyze object detection systems
  *   New evaluation procedures for better understanding object detection performance
  *   Examinations of current evaluation procedures
  *   New datasets designed to examine specific challenges in object detection
  *   New detection methods that provide contributions/insights unrewarded by current evaluation procedures (e.g. improved detector calibration, probabilistic object detection, etc.)

Probabilistic Object Detection (PrOD) Challenge:

Our challenge requires participants to detect objects in video data (from high-fidelity simulation). As a novelty, our evaluation metric rewards accurate estimates of spatial and semantic uncertainty using probabilistic bounding boxes. We developed a new probability-based detection quality (PDQ)<https://arxiv.org/abs/1811.10800> evaluation measure for this challenge, please see the arxiv paper for more details.

Prize money of $2,500 USD to be split between high-performing competitors.

Competitors must submit their solutions to our CodaLab evaluation server (https://competitions.codalab.org/competitions/20597) and submit a paper for this workshop to be considered for the prize money. See the CodaLab page for more details on competing in the challenge.

Regards,

Dr David Hall


Research Fellow

Robotic Vision Benchmarking and Evaluation Project

Australian Centre for Robotic

QUT Centre for Robotics

Queensland University of Technology


e-mail: d20.hall at qut.edu.au

Phone: +61 7 31380656

ORCiD: 0000-0002-5520-0128<https://orcid.org/0000-0002-5520-0128>

Website: https://sites.google.com/view/davidhallcv/home
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20200728/f1694ea2/attachment.html>