WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models

Five images are presented below.
A cue word and number of associated images are given.
Choose the images that best associates with the cue word.

Cue
Associations
2
👉
Choose here!
Go to Challenge the AI page to play the associations games against a rival AI
3.5K+
Associations
<52%
Model Success Rate
>90%
Human Successes Rate

Paper