[visionlist] Release of datasets for model evaluation

[Fritz Günther]

Dear colleagues,

we would like to inform you that we recently released a set of 
gold-standard datasets that you might find interesting for evaluating 
your vision models against human data. More specifically, we have:

• ratings of *visual* similarity for 3,000 word pairs (i.e., how similar 
their referents *look like*), collected from 480 participants
• ratings of visual similarity for 3,000 image pairs, collected from 480 
participants
• ratings of image typicality (how typical is an image for a given 
category) for 7,500 word-image pairs (5 different images per word), 
collected from 902 participants

In addition, if you are interested in human visual processing, we also 
have

• response times and accuracies from a discrimination task (participants 
have to decide whether two images presented in rapid succession are 
identical or not; slower responses for more similar images) for 6,000 
image pairs (3,000 pairs in both presentation orders), collected from 
750 participants
• response times and accuracies from a priming task (two images 
presented one after the other; participants have to decide whether the 
second image is a real or scrambled one; faster responses for more 
similar images) for 6,000 image pairs (3,000 pairs in both presentation 
orders), collected from 750 participants

You can find all datasets and source lists for the images used in the 
studies here: https://osf.io/qvw9c/
The repository also contains links to the paper in which we present the 
datasets in more detail: https://doi.org/10.1037/rev0000392 (or a 
publicly accessible preprint version here: https://psyarxiv.com/n4dmq/). 
The paper contains more detailed information on how we collected and 
pre-processed the data, and a first model validation.

If you have any comments or questions, we are happy to hear from you!

Best wishes,
Fritz Günther, Marco Marelli, Sam Tureski, and Marco Alessandro Petilli


-- 
Dr. Fritz Günther
Junior Research Group Leader
Computational Modelling Lab
Department of Psychology
Humboldt-Universität zu Berlin

www.fritzguenther.de