<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>ECCV 2020 Workshop - Beyond mAP: Reassessing the Evaluation of Object Detectors</b></div>
<div dir="ltr">
<div>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><br>
</b></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Dear Researchers,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Not long now before papers are due for our ECCV workshop. <br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<a href="https://nikosuenderhauf.github.io/roboticvisionchallenges/eccv2020">https://nikosuenderhauf.github.io/roboticvisionchallenges/eccv2020</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview_0 _EReadonly_1"></div>
<br>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
This workshop assesses current evaluation procedures for object detection, highlights their shortcomings and opens discussion for possible improvements.<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Below are some updates on the workshop for if you are interested in submitting to the workshop, competing in the attached probabilistic object detection (PrOD) challenge, or just attending the workshop itself.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<u><b>Workshop Interactive Sessions + Invited Speakers:</b></u><br>
</div>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><br>
</b></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="color:black">Submit questions for discussion by our panel/community through our Slido page</div>
<div style="color:black"><a href="https://app.sli.do/event/v4q9zeul" target="_blank" rel="noopener noreferrer">https://app.sli.do/event/v4q9zeul</a></div>
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>Session 1 - 00:00 - 02:00 UTC+1 August 28th</b></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<ul>
<li>Assistant Prof. Walter Scheirer (University of Notre Dame)</li><li>Dr Larry Zitnick (Facebook AI Research)</li><li>Dr David Hall (Queensland University of Technology)</li></ul>
<div><b>Session 2 - 08:00 - 10:00 UTC+1 August 28th</b></div>
<div>
<ul>
<li>Assistant Prof. Emre Akbas (Middle East Technical University)</li><li>Dr David Hall (Queensland University of Technology)</li></ul>
<div><br>
</div>
<div><u><b>Important Dates:</b></u><br>
</div>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<ul>
<li><b>05 August 2020 </b>Final PrOD submissions to the Evaluation server via CodaLab</li><li><b>07 August 2020</b> Paper Submissions due (Workshop and Challenge) via <a href="https://cmt3.research.microsoft.com/BMREOD2020" target="_blank" rel="noopener noreferrer" title="https://cmt3.research.microsoft.com/BMREOD2020">
CMT</a></li><li><b>12 August 2020 </b>Winner Announcements and Workshop Invitations</li><li><b>15 August 2020 </b>Videos and photo-ready papers for workshop due</li><li><b>28 August 2020 </b>ECCV Workshop Interactive sessions</li></ul>
<div><u><b>Paper Topics of Interest (guidelines):</b></u></div>
<div>
<ul>
<li>New evaluation measures/metrics for object detection</li><li>New evaluation/visualization tools to analyze object detection systems</li><li>New evaluation procedures for better understanding object detection performance</li><li>Examinations of current evaluation procedures</li><li>New datasets designed to examine specific challenges in object detection</li><li>New detection methods that provide contributions/insights unrewarded by current evaluation procedures (e.g. improved detector calibration, probabilistic object detection, etc.)</li></ul>
<div><br>
</div>
<div><u><b>Probabilistic Object Detection (PrOD) Challenge:</b></u></div>
<div><br>
</div>
<div>Our challenge requires participants to <strong>detect objects in video</strong> data (from high-fidelity simulation). As a novelty, our evaluation metric rewards accurate estimates of
<strong>spatial and semantic uncertainty</strong> using probabilistic bounding boxes. We developed a new
<a href="https://arxiv.org/abs/1811.10800" target="_blank" rel="noopener noreferrer">
probability-based detection quality (PDQ)</a> evaluation measure for this challenge, please see the arxiv paper for more details.</div>
<div><br>
</div>
<div>Prize money of <b>$2,500</b> <b>USD</b> to be split between high-performing competitors.</div>
<div><br>
</div>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Competitors must submit their solutions to our CodaLab evaluation server (<a href="https://competitions.codalab.org/competitions/20597" target="_blank" rel="noopener noreferrer">https://competitions.codalab.org/competitions/20597</a>) and submit a paper for
this workshop to be considered for the prize money. See the CodaLab page for more details on competing in the challenge.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div></div>
Regards,</div>
<div><br>
<b></b></div>
<div><b>Dr David Hall</b>
<div>
<div>
<div dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
<br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Research Fellow</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Robotic Vision Benchmarking and Evaluation Project</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Australian Centre for Robotic <br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
QUT Centre for Robotics<br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Queensland University of Technology</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
<br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
e-mail: d20.hall@qut.edu.au</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Phone: +61 7 31380656</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
ORCiD: <span></span><a href="https://orcid.org/0000-0002-5520-0128" target="_blank" rel="noopener noreferrer"><span>0000-0002-5520-0128</span></a><br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0; margin-bottom:0">
Website: <a href="https://sites.google.com/view/davidhallcv/home" target="_blank" rel="noopener noreferrer">https://sites.google.com/view/davidhallcv/home</a></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>