What is the Gemini Interactions API for agents?

The Gemini Interactions API keeps agent context on Google's servers behind an interaction ID, so callers stop hand-managing thought signatures between turns. It also returns typed audio, image and text outputs and supports mixing built-in tools with custom functions in one call. Managed Agents adds a persistent remote sandbox identified by an environment ID.

What the Gemini Interactions API changes for agent workloads

The Gemini Interactions API is Google DeepMind's replacement for the older request-and-response shape of the Gemini API, designed for workloads that run for minutes and span many tool calls. Ivan Leo, a developer experience engineer at Google DeepMind, presented it at the AI Engineer World's Fair 2026 in San Francisco. It stores context server-side and returns an interaction ID instead of asking callers to thread opaque state between turns.

The problem it addresses is scale and shape. Leo's framing in the talk was that models moved from single completions to function calling to agents, and the older endpoints could not comfortably carry that third stage. Deeply nested response objects and separate endpoints for models, image generation and long-running research agents made first-party integration work harder than it should have been.

The Gemini Interactions API consolidates those paths. A single call can produce an image, generate audio, call a search tool, call your own function, or start a research run. Outputs are demarcated by a type discriminator rather than buried in nested objects, so client code parses output.type instead of memorizing a path into a completions payload.

That consolidation matters most for pipelines that chain modalities. Leo demonstrated generating an image, feeding the same interaction forward, and then producing video with a different model while keeping the original context. The interaction ID, not a reassembled message array, is what carries the state between those steps.

Server-side state, interaction IDs and thought signatures

An interaction ID is the identifier the Gemini Interactions API returns on the first turn and accepts back as previous_interaction_id on later turns. Passing it returns the model to the same preserved context, which removes the need to hand-manage the opaque reasoning tokens that Gemini models emit alongside function calls.

Those tokens are thought signatures. Leo described them as an opaque series of values the model passes back, and said that if you do not return them, performance drops. The failure mode he cited from conversations with startups was mundane: manually threading that state through a message array, then losing it to an accidental whitespace character and quietly degrading the model's output.

Server-side state changes the contract. Instead of the caller reconstructing the full conversational and reasoning history on every request, the API holds it and the caller holds a short identifier. That is a smaller surface for bugs, and it is the mechanism the rest of the API design depends on.

The caveat is that this is a stateful service dependency. Context lives on Google's side while the interaction is retained, so the interaction ID is effectively a capability for that conversation. Teams with strict data-handling requirements should confirm retention and residency terms for their configuration rather than assuming a local equivalent.

The steps data model and strongly typed multimodal output

The steps data model replaces the legacy outputs array with a discriminated structure that records what happened at each stage of a turn. Leo described it as a tight discriminator that makes clear, step by step, what the model output, what the thought signatures were, what functions were called, and what content was generated.

The practical benefit is parsing. In the older shape, audio came back inside a nested completions object that Leo admitted he could not recall off the top of his head. With typed outputs, an audio result is marked as audio and an image result is marked as image, so a client switches on the type instead of walking a deep object graph.

Mixing tools in one call follows from the same design. The example in the talk gave an agent the Google Search tool, a URL context tool that retrieves information from web pages, and a custom file_incident function, then let the model decide which to use. The model reasons about what information it needs and returns an answer without the caller orchestrating each hop.

This is where the Gemini Interactions API differs most from a chat completions endpoint. A steps model assumes async tool calls and multiple components working together, which is the structure agent traces actually have once they run for more than a few seconds.

Managed Agents, Antigravity and the environment ID

Managed Agents run the Antigravity harness in a persistent remote sandbox, and the environment ID is what routes a later call back to that same sandbox. The Antigravity agent is the same harness Google uses inside AI Studio and its own Antigravity IDE, which is a claim about consistency between local and cloud runs rather than about a separate cloud-only agent.

Two primitives carry the workload. The interaction ID preserves conversational context, and the environment ID preserves the machine state: installed packages, created files, downloaded dependencies. Leo described a demo where a repository analysis continued across calls, with the model finding the same files exactly where it left them.

The showcase was a hackathon repository containing a custom programming language built from scratch, complete with a web page, schemas and a reinforcement learning training loop. Leo said the run consumed over two million tokens while analyzing it. That figure comes from the speaker's own demo, so treat it as a scale indicator for how long these runs can go rather than a benchmark.

Sandbox configuration accepts loadable sources. Google Cloud Storage buckets, GitHub repositories and inline files are all supported, so custom dependencies and helper files can be dropped into the environment before the agent starts working.

Skills, the credential proxy and named agents

A man-in-the-middle proxy inspects outbound calls from the sandbox and injects credentials at the network boundary, so the agent executes code against an API without ever holding the token. Leo described the case where every outbound call to the GitHub API gets a token injected dynamically, meaning a prompt-injected agent that tries to leak a credential has nothing to leak.

Skills travel between local and remote. Workflows you tuned in the Antigravity IDE can be packaged into a folder and uploaded through the same sources mechanism, giving the remote agent the same skills and harness prompt as the local one. Leo's claim is parity of harness and prompt, not identical behavior in every environment.

Named agents are the packaging step. Google offers two paths: freeze an agent with a configured set of files and network sources, or chat with an agent iteratively until the environment has the dependencies you want, then save it. Leo said the platform supports up to a thousand named agents, and that you pay for the model rather than for sandbox storage. Quota and pricing terms change, so verify current limits before capacity planning.

Security claims deserve care here. A credential-injecting proxy and sandbox isolation support an architecture where the model never sees secrets, but they do not by themselves establish compliance with HIPAA, financial regulation, or any specific tenant-isolation standard. Those remain application and organizational controls.

Migrating to the Interactions API with the Gemini API CLI

The Gemini API CLI is the open-source command-line tool for testing Gemini models locally, and it packages a local project into a folder that can be uploaded to create an agent. Leo presented it as the bridge between iterating locally and deploying remotely: build and test in the CLI or an Antigravity agent, then ship the folder.

Google also shipped an Interactions API skill for coding agents. The documented purpose is migration assistance, and Leo tied it to a familiar annoyance: coding assistants that keep reaching for older model names such as Gemini 2.5 Flash or Gemini 2.0. The skill carries a current list of models and methods so the assistant targets the right surface.

He also stated that these skills are evaluated regularly to confirm they do what they claim, a process he attributed to a colleague. That is an internal evaluation claim, not an external audit, and it should be read as vendor-reported.

Treat every model name in this section as time-sensitive. Specific Gemini version numbers mentioned in a recorded talk age quickly, so check the current model list before pinning anything in production. Leo's stated direction was that new models land on the Interactions API surface, which makes the migration path a forward-looking decision as much as a cleanup.

How the two API generations compare

The table below compares the older Gemini API shape described in the talk with the Gemini Interactions API on the dimensions the talk actually covered. Every row reflects what the speaker said about the transition, not an independent benchmark.

Frequently asked questions about the Gemini Interactions API

• What is the Gemini Interactions API? It is Google DeepMind's API surface for agent workloads, replacing the older Gemini API request-and-response shape. It keeps conversation context server-side behind an interaction ID, returns strongly typed multimodal outputs, and lets one call mix built-in and custom tools. Ivan Leo presented it at the AI Engineer World's Fair 2026.

• What is an interaction ID used for? An interaction ID identifies a stored conversation context on Google's servers. You pass it back on later turns as previous_interaction_id, and the model resumes with the same reasoning state, including the thought signatures it produced earlier.

• Why did Gemini stop requiring manual thought signature handling? Thought signatures are opaque values the model returns alongside reasoning and tool calls. Handling them by hand was error-prone, and Leo cited startups losing context to a stray whitespace character. Server-side state removes that manual step, and Leo said omitting the signatures reduces model performance.

• What is an environment ID in Managed Agents? An environment ID routes a call back to a specific persistent remote sandbox. It preserves installed packages, created files and downloaded dependencies across turns, which is how a repository analysis can continue without re-creating the workspace.

• Can Managed Agents keep an API credential secret from the model? Google implements a man-in-the-middle proxy that injects credentials into outbound requests at the network boundary. Leo's example replaces the GitHub token on every outbound call, so the sandboxed agent executes code against the API without the token being visible to it.

• How do skills move from a local Antigravity setup to the cloud? Package the tuned workflow into a folder and upload it through the sources mechanism, which accepts Google Cloud Storage buckets, GitHub repositories and inline files. Leo said this gives the remote agent the same skills and harness prompt as the local run.

• Is the Gemini API CLI open source? Leo described the Gemini API CLI as newly open sourced, and it is published on GitHub. It is a wrapper for testing Gemini models from a terminal and for packaging a local coding agent project into an uploadable folder.

• Does the Gemini Interactions API replace the older endpoints? Leo said new Gemini models land on the Interactions API surface, and Google provides a migration skill for coding agents. Teams still running older endpoints should confirm current deprecation timelines against Google's documentation rather than the recording.

• What does the steps data model return? It returns a discriminated sequence describing each stage of a turn: model output, thought signatures, function calls and generated content. Clients switch on the type discriminator for audio, image or text instead of walking a deeply nested object.

Turning a conference talk into a written reference

Ivan Leo's talk works as a talk because the argument builds in sequence: completions, then function calling, then agents, then the API shape that agents actually need. That sequence is exactly what gets lost when a 17-minute recording sits on YouTube and the documentation only shows the endpoint that survived.

If you hold knowledge like this, conference sessions, interviews, internal walkthroughs, customer conversations, you already have the hard part done. The Gemini Interactions API skill and the Crazystack Typescript and Bootcamp do Dev Doido communities are good examples of people who teach in one medium and could reach further in another. The friction is the writing.

Skala Blog turns a YouTube video into a draft article. You paste the URL, the video gets transcribed, and the transcript becomes structured text you can edit and publish. The talk above is a live demo of a topic like this, and Crazystack maintains tooling worth checking alongside it. For a speaker like Gustavo Dev Doido, the recording can become the reference document.

A practical note on scope. Neither a transcript nor an API announcement covers retention terms, current pricing or the published deprecation schedule for the endpoints being replaced. Verify those against Google's own documentation before you commit a migration plan, and keep the original recording as your source for what the speaker actually said.

Source video