Add Agents window startup A/A experiment trigger - #328454
Merged
Benjamin Christopher Simmonds (benibenj) merged 2 commits intoAug 4, 2026
Merged
Conversation
Read agentsWindowStartupAA at BlockStartup when the Agents window first starts so ExP can measure A/A assignment without changing UX. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4d39cd2-7ec9-4170-a2e6-e3de5579589d
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an Agents window startup A/A experiment exposure read.
Changes:
- Reads
agentsWindowStartupAAduring startup. - Registers the contribution for desktop Agents windows.
- Adds unit coverage and documentation.
Show a summary per file
| File | Description |
|---|---|
src/vs/sessions/SESSIONS.md |
Documents the startup experiment. |
sessionsWindowStartupExperiment.test.ts |
Verifies the treatment read. |
sessionsWindowStartupExperiment.ts |
Implements the experiment contribution. |
chat.contribution.ts |
Registers the desktop startup contribution. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Balanced
| } | ||
|
|
||
| registerWorkbenchContribution2(SelectAgentsFolderContribution.ID, SelectAgentsFolderContribution, WorkbenchPhase.BlockStartup); | ||
| registerWorkbenchContribution2(SessionsWindowStartupExperiment.ID, SessionsWindowStartupExperiment, WorkbenchPhase.BlockStartup); |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4d39cd2-7ec9-4170-a2e6-e3de5579589d
Contributor
Author
Product-ready proof videos (visible cohort assignment)A/A arms are UX-identical; proof is the cohort allocation token visible in Developer: Show Telemetry (Find + full-window HEVC recording). Treatment
Control
Notes
|
Member
|
Ralph Feltis (@rfeltis) are you working with anyone on our team to land this? |
Contributor
Author
|
Hey Connor Peet (@connor4312) , I'll be working with Benjamin Christopher Simmonds (@benibenj) to land this. I'll probably need 2 stamps. Is this the right place to trigger an experiment on Agents window start? |
Connor Peet (connor4312)
assigned
Benjamin Christopher Simmonds (benibenj) and unassigned
Connor Peet (connor4312)
Ralph Feltis (rfeltis)
marked this pull request as ready for review
August 3, 2026 23:46
Benjamin Christopher Simmonds (benibenj)
approved these changes
Aug 4, 2026
Benjamin Christopher Simmonds (benibenj)
enabled auto-merge (squash)
August 4, 2026 15:35
TylerLeonhardt
approved these changes
Aug 4, 2026
Vritant Bhardwaj (vritant24)
approved these changes
Aug 4, 2026
Benjamin Christopher Simmonds (benibenj)
merged commit Aug 4, 2026
443507c
into
microsoft:main
29 checks passed
Don Jayamanne (DonJayamanne)
added a commit
that referenced
this pull request
Aug 4, 2026
* origin/main: (31 commits) Improve workspace picker preselection (#328995) agentHost: support Codex custom agents and runtime enablement (#328956) Finalizes customEditorPriority proposal. Closes #292379 (#329002) Add Agents window startup A/A experiment trigger (#328454) sessions: show created session pill in response summary (#328984) Fix onboarding microphone picker visibility (#329011) Explains how to develop the markdown editor (#329009) Conditional agent-window auth for signed-out users (#328990) Fix BYOK enterprise policy handling in agent host Agent Host changes for fix/agent-host-byok-enterprise-policy agentHost: drive tool execution from the session input queue (#328989) Make Integrated Browser smoke tests deterministic across build qualities (#328983) Accept box sizing screenshot changes Avoid large Component Fixtures step outputs Remove component fixture box sizing reset fix: guard stale line numbers in test decorations (fixes #328988) sessions: fix maximized side pane toggle (#328974) Add component fixture rendering controls Reduce floating panel margins for layout consistency (#328963) agentHost: support file completions across workspace roots (#328944) ... # Conflicts: # src/vs/sessions/SESSIONS.md # src/vs/sessions/contrib/providers/agentHost/browser/baseAgentHostSessionsProvider.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
agentsWindowStartupAAwhen the Agents window first starts.Experiment
dbd12384-5001-4e90-a863-526321eaf233/vscodeexpws~agentswindowtestagentsWindowStartupAA)Test plan
tasClientReadTreatmentCompletewithtreatmentName=agentsWindowStartupAAand assignment context tokens for each armProduct-ready proof videos (visible cohort assignment)
A/A arms are UX-identical; proof is the cohort allocation token visible in Developer: Show Telemetry (Find + full-window HEVC recording).
Treatment
agents-window-startup-aa-variant:31559566Control
agents-window-startup-aa-control:31559570Notes