Skip to content

fix: gRPC retry on PROTOCOL_ERROR for IDB client stability - #30

Merged
mattleibow merged 13 commits into
mainfrom
pr/grpc-retry
Feb 27, 2026
Merged

mattleibow merged 13 commits into
mainfrom
pr/grpc-retry

Conversation

@mattleibow

Copy link
Copy Markdown
Collaborator

Adds WithRetryOnProtocolErrorAsync helper that reconnects on HTTP/2 PROTOCOL_ERROR, fixing intermittent failures where the connection to idb_companion v1.1.8 gets poisoned on CI.

Changes:

  • AppleDev.FbIdb/IdbClient.cs: Added retry wrapper applied to all public gRPC methods (DescribeAsync, FocusAsync, SetLocationAsync, ListAppsAsync, ListTestBundlesAsync, ListCrashLogsAsync)
  • AppleDev.FbIdb.Test/IdbClientIntegrationTests.cs: Skip ListCrashLogsAsync due to upstream idb_companion crash (Get always 'Connection lost' when execute 'idb crash list' meta/idb#868)

Context:
ScreenshotAsync was already 100% stable on CI because it had an ad-hoc PROTOCOL_ERROR catch with simctl fallback. All other methods lacked this handling, causing 40-60% failure rates on CI. This generalizes the pattern into a reusable retry helper.

mattleibow and others added 5 commits February 27, 2026 04:49
Add WithRetryOnProtocolErrorAsync helper that reconnects on HTTP/2
PROTOCOL_ERROR, fixing intermittent failures where the connection to
idb_companion v1.1.8 gets poisoned. Applied to DescribeAsync,
FocusAsync, SetLocationAsync, ListAppsAsync, ListTestBundlesAsync,
and ListCrashLogsAsync.

Also skip ListCrashLogsAsync test due to upstream idb_companion crash
(meta/idb#868).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace SkipIfNotReady() with SkipIfNotSupported() that only skips
on non-macOS or when idb_companion is not installed. Simulator boot
failures now throw instead of silently skipping. The only remaining
hard skip is ListCrashLogsAsync due to idb_companion server crash
(meta/idb#868) which retries cannot fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The simulator boot can timeout on CI runners. Keep Skip.If for
environment issues (not macOS, no idb, boot timeout) to avoid
false failures. Only test-specific skip is crash_list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ScreenshotAsync now uses WithRetryOnProtocolErrorAsync first, then
falls back to simctl only if all retries are exhausted. Previously
it fell back immediately on first PROTOCOL_ERROR, which could return
empty data from simctl on CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mattleibow and others added 8 commits February 27, 2026 10:00
Replace dorny/test-reporter with EnricoMi/publish-unit-test-result-action
which posts test summary comments on PRs with pass/fail/skip counts.
Uses the /macos variant since CI runs on macos-latest.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove IsRunningInCI() skip from all SimCtlComplexTests — these
  should run on CI, not be silently skipped
- Increase simulator boot timeout from 120s to 300s in IDB fixture
- Increase idb_companion startup timeout from 60s to 120s
- Increase SimCtlComplexTests boot timeout from 60s to 300s

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Maps predicate was unreliable on CI — Maps may not generate logs
on freshly booted simulators. SpringBoard always runs and produces
logs on any booted iOS simulator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Plain text logs include headers/multi-line entries without process
  name; use Assert.Contains instead of Assert.All
- Increase ObserveAnyDevices timeout from 10s to 30s for CI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
idb_companion can crash on CI when xcode-select is slow (internal 10s
timeout in idb_companion v1.1.8). Catch TimeoutException from
companion startup and skip the test instead of failing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dicate tests

- Format WithRetryOnProtocolErrorAsync calls as single lines
- Restore [SkippableFact] on all SimCtlComplexTests
- Restore Maps predicate tests: launch Maps first so logs are
  guaranteed to exist, then filter and verify

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mattleibow
mattleibow requested review from Redth and Copilot and removed request for Copilot February 27, 2026 18:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds automatic retry logic for gRPC PROTOCOL_ERROR failures in the IDB client to fix intermittent connection issues with idb_companion v1.1.8 on CI, which were causing 40-60% failure rates. It also increases test timeouts and removes CI-specific test skips to improve test reliability.

Changes:

  • Added WithRetryOnProtocolErrorAsync helper that automatically reconnects on HTTP/2 PROTOCOL_ERROR and retries up to 3 times
  • Applied retry wrapper to 6 public gRPC methods: DescribeAsync, FocusAsync, SetLocationAsync, ListAppsAsync, ListTestBundlesAsync, and ScreenshotAsync
  • Increased test timeouts for simulator operations (boot wait: 60s→300s, IDB startup: 60s→120s, device observation: 10s→30s)
  • Removed CI-specific test skips in SimCtlComplexTests
  • Improved log filtering tests by explicitly launching Maps app
  • Skipped ListCrashLogsAsync test due to upstream idb_companion crash bug

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
AppleDev.FbIdb/IdbClient.cs Added retry helper and applied it to 6 gRPC methods; improved error handling for PROTOCOL_ERROR
AppleDev.FbIdb.Test/IdbClientIntegrationTests.cs Increased startup timeout, added timeout exception handling, skipped ListCrashLogsAsync test due to upstream bug
AppleDev.Test/SimCtlComplexTests.cs Increased boot timeout, removed CI skip checks, improved log filtering tests by launching Maps app
AppleDev.Test/XCDeviceTests.cs Increased device observation timeout from 10s to 30s

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread AppleDev.FbIdb/IdbClient.cs Resolved
Comment thread AppleDev.FbIdb/IdbClient.cs Resolved
@mattleibow
mattleibow merged commit c0fd026 into main Feb 27, 2026
7 checks passed
@mattleibow
mattleibow deleted the pr/grpc-retry branch February 27, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants