Google DeepMind launches agentic video understanding for Gemini

Sep. 1, 2026

Google DeepMind has launched agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. The company says the new feature allows the model to dynamically scan video segments, improving accuracy while cutting token usage by up to 88% and costs by up to 66%.

The capability is available today for video uploads and YouTube videos through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. Developers can enable it by setting API processing to "agentic".

Rohan Doshi, Senior Product Manager at Google DeepMind, and Mario Lučić, Research Director at Google DeepMind, said the new agentic feature for video analysis can cut token consumption by up to 88%, reduce costs by up to 66%, and boost quality by up to 7%.

According to Google DeepMind, agentic video understanding is similar to agentic vision, which combines code execution with Gemini models’ native image understanding. The company said the video tool uses Gemini’s native video capabilities to improve performance and support use cases including sub-second moment retrieval, more accurate anomaly detection, precise counting, and other video-processing tasks. That matters most for developers working with long-form clips, where fixed-rate processing can miss short-lived details or force higher token use.

Benchmarks

Google DeepMind said current static processing typically ingests video at a fixed frames-per-second rate, with a default of 1 FPS that can be adjusted through the API. By contrast, agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect targeted video segments across visual frames, audio, and transcripts.

Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%, the company said.

The company said these gains are especially notable for long-form video, including 10-minute how-to guides, 90-minute lectures, and multi-hour recordings, where static processing forces developers to choose between high token costs and methods that can omit important details.

Google DeepMind said activating agentic video understanding reduces token consumption by up to 88% and increases accuracy by up to 7% with Gemini 3.7 Flash. The company added that, while the gains apply across all three supported models, Gemini 3.7 Flash with agentic understanding offers the best overall quality and the strongest balance of quality and cost efficiency among the models tested for video understanding.

How it works

Instead of static processing, where the model ingests media streams at a fixed frame rate, agentic video understanding lets Gemini decide what to watch, how quickly to watch it, and whether to use frames, audio, or transcript data. The system fetches only the moments and signals needed for the task.

Google DeepMind said developers could previously perform this process manually, but with agentic video understanding, Gemini can handle it through an agentic loop by invoking an internal tool to load the relevant part of the video file. The company said this significantly reduces development overhead.

Capabilities and use cases

Google DeepMind said agentic video understanding changes how developers can process long-form video content across a range of demanding applications.

• Sub-second moment retrieval: Pinpoint split-second state changes and tight cut boundaries that are easy to miss at 1 FPS, enabling precise automated video editing.
• Long-form needle-in-a-haystack search: Answer complex questions across multi-hour videos without consuming millions of tokens.
• Anomaly detection: Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts.
• Counting action & object: Accurately track repeated physical movements and distinct objects over time.

Token-efficient long-form video analysis

Google DeepMind said Gemini 3.7 Flash shows large token reductions and accuracy improvements with agentic video understanding on LongVideoBench, a long-form video understanding benchmark. For teams building review, moderation, search, or analytics tools, those gains could reduce the tradeoff between breadth of coverage and the cost of inspecting every frame.

Accurate fast action analysis with dynamic FPS

With agentic video understanding, 3.7 Flash can accurately count a fast-paced movement by scanning and rewatching the video at different frames per second as needed.

Token-efficient needle-in-a-haystack search

Using agentic video understanding, Gemini 3.7 can answer complex questions based on video content while consuming significantly fewer tokens than static analysis, Google DeepMind said.

Real-world results

Google DeepMind said many early access partners reported strong performance while testing agentic video understanding.

Getting started

Agentic video understanding is available through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. Google DeepMind said it uses standard Gemini API token pricing with no additional feature fee.

To enable the feature, developers should set processing to "agentic" in the API configuration. Google DeepMind also directed readers to its developer guide for more information on how to get started.

The company said it is also bringing the efficiency and quality improvements of agentic video understanding to users across Google products. The feature will roll out soon to all users in the Gemini app across Flash and Flash-Lite models. In the coming months, Google DeepMind said agentic video understanding will also power YouTube’s “Ask YouTube” feature on the video watch page, using Gemini to deliver higher-quality answers grounded in the visuals.

Google DeepMind acknowledged the contributions of Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team.