Skip to content

feat(agent): add LiveLlmProvider abstraction and Gemini Live WebSocket provider - #69

Open
juliantoledo wants to merge 2 commits into
android:mainfrom
juliantoledo:feat/live-llm-provider
Open

juliantoledo wants to merge 2 commits into
android:mainfrom
juliantoledo:feat/live-llm-provider

Conversation

@juliantoledo

@juliantoledo juliantoledo commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Part 2 of 5 in the Gemini Live voice assistant series (following #65):

  1. PR 1: Refactor tool discovery and execution (refactor(agent): unify AppFunction discovery, input conversion, and execution #65 — merged)
  2. PR 2 (this PR): Live LLM provider layer (LiveLlmProvider abstraction + GeminiLiveProviderImpl)
  3. PR 3: Full-duplex audio capture/playback (LiveAudioSession + GeminiLiveAudioHelper)
  4. PR 4: Live session orchestration (LiveAgentOrchestrator)
  5. PR 5: Android VoiceInteractionService and Live UI integration

Changes in this PR

  • LiveLlmProvider / LiveLlmSession / LiveLlmEvent: Introduce a provider-agnostic bidirectional streaming abstraction returning LiveLlmSession (with close() and a withSession lifecycle extension), supporting configurable input/output sample rates, input/output transcriptions, turn completion, interruption events, tool call requests/responses, and LiveLlmEvent.Error.
  • LlmProviderFactory / LlmProviderFactoryImpl: Extend the provider factory with getLiveProvider(name: LlmProviderName): LiveLlmProvider.
  • GeminiLiveProviderImpl & GeminiLiveVoice: Implement Gemini Live over the BidiGenerateContent WebSocket endpoint (models/gemini-3.8-live) using Ktor WebSockets (with pingIntervalMillis configured in DataModule), supporting prebuilt studio voice selection (GeminiLiveVoice persisted in SettingsRepository), realtime PCM audio streaming, input/output audio transcriptions, and tool call/response frames.
  • GeminiLiveToolConverter & ToolConverter / JsonExtensions: Add GeminiLiveToolConverter (normalizing OpenAPI schema fields for BidiGenerateContent), provider-agnostic selectDistinctTools, findToolByName, and resolveToolName extensions on ToolConverter, and shared toPrimitive() / toArgumentsMap() JSON helpers in domain/JsonExtensions.kt.
  • Unit Tests: Add unit tests in GeminiLiveProviderImplTest, GeminiLiveToolConverterTest, ToolConverterTest, and SettingsRepositoryTest.

Test Plan

  • ./gradlew spotlessCheck testStandardDebugUnitTest testRetailDebugUnitTest compileStandardDebugAndroidTestKotlin compileStandardDebugScreenshotTestKotlin
@juliantoledo
juliantoledo requested a review from a team as a code owner November 1, 2026 18:23
@juliantoledo
juliantoledo force-pushed the feat/live-llm-provider branch from d2c1af9 to e52f370 Compare November 1, 2026 21:23

@mingweiliao mingweiliao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants