Repository navigation
Conversation
…s stop on When every credential a request may use is cooling down after Codex's 429 usage_limit_reached, the proxy answered model_cooldown with a Retry-After as long as the quota window. Codex recognises a usage limit only by error.type, so it treated the refusal as a rate limit: over WebSocket the connection closed without an event five times, then over HTTPS it slept until Retry-After, hours or days, before retrying. Such a pool now gets, while its first reset is more than five minutes away: - the model_cooldown body plus type usage_limit_reached, resets_at, retryable false and limit_window_minutes; - X-Should-Retry: false and Codex's rate-limit headers for the window that reopens first, instead of Retry-After; - the same refusal as a WebSocket error event instead of a silent close. The evidence is the usage_limit_reached refusal each credential's cooldown recorded, which survives a restart. A pool with any other 429, and a short cooldown, keeps today's retryable response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Documents the change in FORK.md, adds hook checks to fork-check.sh and TestFork* wrappers so a sync cannot drop it silently while router-for-me#7184 is open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h wrapping The Codex executor stores usage_limit_reached under error, at the top level, under body.error (WebSocket) and response.error (SSE); the evidence check read only error. A usage limit wrapped in another error now answers with the cooldown's own body, so error.type survives on HTTP and WebSocket. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Update:
Verified live on rust-builder with Codex CLI 0.166.0 and an isolated T3 dev server:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Codex threads in T3 never show Limited when both ChatGPT logins are out of quota, and sit "running" for hours or days instead. While every Codex credential cools down on
usage_limit_reached, the Gateway answers upstream'smodel_cooldown429 withRetry-Afterset to the quota window. Codex 0.166 recognises a usage limit only byerror.type, so it retries:Retry-After.On rust-builder one turn waited 38½ hours, from 2026-10-31 02:10Z to 2026-11-01 16:49Z. Investigation: factory-notes
incidents/2026-11-01-codex-quota-stops-not-limited.md.This is not caused by a fork patch. #10's pass-through matches only Meta's refusal shape.
What
This is the change proposed upstream as router-for-me#7184, cherry-picked onto
main. Theselector.goconflict with #10'srefusalfield was resolved by keeping both. While a Codex pool's first reset is more than five minutes away:model_cooldownplustype: "usage_limit_reached",resets_at,retryable: falseandlimit_window_minutes, the same shape as fix(auth): report explicit quota exhaustion as a final usage limit router-for-me/CLIProxyAPI#6647;X-Should-Retry: falseandX-Codex-Primary-*(5-hour window) orX-Codex-Secondary-*(weekly), instead ofRetry-After;errorevent instead of a silent close.Fork-only commits:
fork-check.shhook checks andTestFork*wrappers, so a sync cannot drop this while fix(auth): report exhausted Codex quota as a usage limit Codex clients stop on router-for-me/CLIProxyAPI#7184 is open;gofmtof an untouched upstream test file.Verification
usage_limit_reached.Reconnecting... 2/5–5/5(Connection reset without closing handshake), thenReconnecting... 1/5 (exceeded retry limit, last status: 429)and aRetry-After: 10640sleep, still waiting at 75 s.You've hit your usage limit. Try again at 7:02 PM.andturn.failedwithin a second, over WebSocket and HTTPS.bash .github/scripts/fork-check.sh: exit 0 (blocking checks and the advisory upstream tests).go test -raceon the new tests passes. Each production change was reverted in turn, and its tests failed.Applying
Merging publishes
:latest. Applying it recreates the Gateway container, which cuts in-flight Codex and Claude requests, T3 threads and Rocky Brain jobs. Apply when idle. No config change is needed.T3 needs no update for this: current T3 already shows Codex's
usageLimitExceededas Limited, and the headers give it the reset time for "Resume at reset". T3's own fix for the current behaviour is being upstreamed separately.🤖 Generated with Claude Code