Vinclat is a Catalan evaluation dataset designed to assess LLMs' reasoning and cultural knowledge through 1,000 human-crafted riddles requiring multi-step inferential and abductive reasoning.
By combining independent clues that must be creatively connected, Vinclat provides a robust benchmark that emphasizes creative reasoning and resists rapid saturation.
Cite
@inproceedings{pamies2026vinclat,
title={Vinclat: Evaluating Reasoning, Cognition and Culture in One Game},
author={Pamies, Marc and Aula-Blasco, Javier and Gonzalez-Agirre, Aitor and Villegas, Marta},
booktitle={Proceedings of the First Workshop on Multilingual Multicultural Evaluation},
pages={49--66},
year={2026}
}