Meta AI researchers have introduced GAMUT, a new benchmark aimed at evaluating a dimension of AI performance that most existing assessments tend to overlook: factual completeness. While widely used factuality benchmarks such as TruthfulQA, FEVER, and HaluEval focus on detecting false or hallucinated claims, they generally do not measure whether a model's answer omits relevant information. GAMUT addresses that gap by testing whether an AI system includes all the relevant facts a comprehensive answer should contain.

The benchmark, whose full name is Grounded Assessment of Multimodal Factuality, was detailed in an arXiv paper published on July 21, 2026. It employs a structured rubric-based system that transforms the question of whether an AI has covered all necessary information into binary, machine-gradable checklists.

How GAMUT Works

GAMUT addresses the problem of information omission through a two-level meta-rubric framework. The system organizes required content hierarchically — not merely listing facts that an answer should include, but structuring them by importance and category. These requirements are then converted into yes-or-no questions that language models can grade consistently and reliably.

The benchmark comprises 1,813 questions grounded in real-world wearable imagery spanning ten distinct domains. Each question is paired with expert-verified rubrics, with human specialists defining the criteria for a complete answer before any AI model was evaluated.

Model Performance Findings

Meta assessed 14 different AI models against the GAMUT benchmark. The highest-scoring model was Gemini 3.1 Pro, which achieved 58.7% — a result that underscores the difficulty of the benchmark and the gap between current model capabilities and full factual completeness.

According to the researchers, GAMUT demonstrated strong discriminative ability across the models tested, meaning it could effectively distinguish stronger performers from weaker ones rather than producing clustered or undifferentiated scores.

Meta has made the benchmark resources publicly available through Hugging Face, enabling other researchers and developers to evaluate their own models against the same rubrics.

Significance for Multimodal AI Evaluation

GAMUT's multimodal design adds a critical dimension to its assessments. Because the benchmark's questions are anchored in wearable imagery, participating models must interpret visual inputs and generate comprehensive textual responses, testing both visual understanding and the ability to produce factually complete answers — capabilities that are increasingly relevant as AI assistants are integrated into devices such as smart glasses and other wearable platforms. Meta itself markets Ray-Ban Meta smart glasses, making factual completeness in visual question-answering a practical concern rather than a purely academic metric.