Google Gemini can now decide which parts of a video to watch — using up to 88% fewer tokens

Google has given Gemini a new way to analyse video that lets the AI decide what it actually needs to watch, instead of processing footage at a fixed rate from beginning to end.

The company launched agentic video understanding on 1 September 2026 for Gemini 3.7 Flash, Gemini 3.6 Flash and Gemini 3.5 Flash-Lite.

It is available for uploaded videos and YouTube videos through the Gemini API in Google AI Studio and Google’s Gemini Enterprise Agent Platform.

Google says its testing found that the new approach can use up to 88% fewer tokens, reduce analysis costs by up to 66%, and improve accuracy by up to 7% compared with conventional video processing.

Those are Google’s benchmark results rather than a promise that every video workload will achieve the same savings.

Gemini does not need to watch every second in the same way

The important change is how Gemini processes video.

By default, conventional Gemini video processing samples footage at a fixed rate — normally one frame per second — and puts that information into the model’s context.

That approach is relatively straightforward, but it means longer videos consume increasingly large numbers of tokens even if only a small section of the footage is relevant to the user’s question.

Agentic video understanding changes this.

Gemini can dynamically navigate through the video’s timeline and decide whether it needs to inspect frames, audio or the transcript.

It can also alter how closely it examines a particular section.

For example, the model could scan through a long recording relatively quickly, identify a potentially important moment and then revisit that short section at a higher frame rate.

Google describes the system as an agentic loop in which Gemini can repeatedly load the parts of the video it needs before producing its answer.

That distinction is particularly important for long videos.

Why fewer tokens matter

AI models do not see an hour-long video in quite the same way a human simply watches it.

Video frames, audio and other information need to be represented as tokens that the model can process.

Google’s documentation says static video processing can use approximately 100 tokens per second at the default low media resolution, or around 300 tokens per second at higher resolution.

That can become substantial over a long recording.

Google gives an illustrative example in its developer documentation where one hour of video could consume roughly 1.08 million tokens using static processing, compared with around 108,000 using agentic processing in the example scenario.

Actual consumption depends on the video and question being asked.

The advantage is therefore not simply that Gemini can analyse video differently.

It potentially avoids paying the computational cost of examining information that is irrelevant to the question.

Google says Gemini can find moments lasting less than a second

The ability to revisit relevant sections also allows Gemini to deal with events that conventional one-frame-per-second sampling might miss.

Google lists sub-second moment retrieval as one of the feature’s main uses.

It could, for example, help identify the precise moment at which something changes in footage rather than returning only a rough timestamp.

Google also highlights anomaly detection.

If Gemini notices something unusual in a particular time window, it can inspect that section at a higher frame rate to look for rapid movement or visual details that might have been missed during the initial scan.

Other stated applications include counting repeated actions or objects and searching for specific information inside hours of video.

Long videos are where the change becomes more interesting

The technology makes considerably more sense when dealing with a two-hour lecture or hours of security footage than when analysing a 20-second video clip.

Google says its efficiency improvements were particularly pronounced on longer content, ranging from 10-minute instructional videos to 90-minute lectures and multi-hour recordings.

For a straightforward question about a short video, the additional agentic navigation can actually introduce some extra time before the model starts answering.

Google’s developer documentation warns that agentic processing may slightly increase time to first token for short clips under five minutes because the model first has to reason about how it will navigate the footage.

That makes the technology less about replacing every existing video-processing method and more about choosing a smarter method for complicated or lengthy footage.

Why it matters in South Africa

Unlike some AI launches that arrive with unclear regional availability, the underlying developer services are available locally.

Google officially lists South Africa as a supported country for both Google AI Studio and the Gemini API.

That gives the technology possible applications well beyond ordinary consumer video search.

South African companies increasingly generate large amounts of footage across security systems, broadcasting, retail, mining, manufacturing and training environments.

Finding a particular event across hours of recordings can be expensive if an AI system has to process every frame at the same level of detail.

A model that first searches broadly and only inspects relevant sections more closely could make some of those workloads less computationally intensive.

Security-camera analysis is an obvious example, although any organisation processing surveillance or other sensitive footage would still need to assess privacy, data-governance and regulatory requirements before sending video to an AI service.

The same technology could also be useful for broadcasters searching archive footage, education providers indexing recorded lectures, and companies analysing training or industrial inspection video.

The feature is coming to ordinary Gemini users too

Agentic video understanding is initially a developer and enterprise feature, but Google does not intend to keep it there.

The company says the technology will roll out to users of the Gemini app across its Flash and Flash-Lite models.

Google also plans to use the technology for Ask YouTube, allowing the feature to provide answers based more accurately on the visual contents of videos.

Google has not provided an exact date for those consumer rollouts, saying Gemini app support is coming soon and Ask YouTube integration will follow in the coming months.

For developers, however, agentic video understanding is available now.

And the bigger change may be conceptual: Gemini is moving from simply accepting a video as input to actively deciding how it needs to watch it.