You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[FR]: Testing agent should handle AppFunctionUriGrant results and pass returned images to the model #63
A function can return an image the standard way, as an AppFunctionUriGrant (a content URI plus FLAG_GRANT_READ_URI_PERMISSION). The Testing Agent doesn't recognize this:
Not shown:AgentOrchestrator.handleLlmResponse (around line 301) only picks up images from a result field named imageUri, the agent's own convention from generateImage (see [FR]: Render image URIs inline within the Test/Sample Agent Chat UI #12). A standard AppFunctionUriGrant result is passed to the model as plain text, and no image appears in the chat.
Not seen by the model: even with imageUri, the image is only attached to the chat message for the user. GeminiProviderImpl never sends image content to the model, so a function's returned image never reaches the LLM.
Why it matters
Visual apps rely on returning renders so the agent can check its own work. For example, a design app can expose renderDesign(): AppFunctionUriGrant and instruct the agent to render after changes and correct what it sees. With the Testing Agent, the model works blind, so this loop can't be tested at all, even though it is exactly the kind of flow a system agent would need to support.
Suggested behavior
Recognize AppFunctionUriGrant (or any result whose URI points to an image/* type) as an image result, alongside the imageUri convention.
Read it through the granted permission and attach it to the next model turn as image input (Gemini inline_data), so the model can see it.
Activity