Make no-op DCE return the input jaxpr, so lowering does not depend on trace history - #40318
Open
kodlan wants to merge 1 commit into
Open
Make no-op DCE return the input jaxpr, so lowering does not depend on trace history#40318kodlan wants to merge 1 commit into
kodlan wants to merge 1 commit into
Conversation
… trace history For jax-ml#40312. Lowering the same jitted function could emit different StableHLO depending on what the process traced earlier. The MLIR lowering cache dedups repeated subcomputations by jaxpr object identity, but dead code elimination rebuilt a new jaxpr even when it eliminated nothing, so that identity was only stable while the right weakref LRU cache entries stayed warm. The DCE caches are layered (the pjit DCE rule has its own cache on top of pe.dce_jaxpr's) and a cache hit refreshes only the outer entry, so unrelated tracing could evict the inner entries while a stale outer entry survived. A later lowering then mixed the stale outer result with freshly recomputed inner ones, saw two structurally equal but distinct body jaxprs, and emitted a duplicate function (7 vs 6 StableHLO functions in the issue's repro, and through XLA inlining a different optimized module on TPU). The fix: when DCE removes nothing (all outputs and inputs used, no equation changed), pe.dce_jaxpr returns the input jaxpr itself instead of an equal copy, anchoring object identity at the unchanged leaves so the lowering dedup for unchanged subjaxprs no longer depends on cache eviction state. Inside the weakref caches the no-op case is stored as a None sentinel rather than the jaxpr: caching the key as its own value would give the entry a strong reference to its weak key, pinning it (and everything the jaxpr closes over) forever. The two caches that re-cache dce results under the same key (the pjit DCE rule's and the closed-call rule's) store the same sentinel. This intentionally covers only subcomputations that DCE leaves unchanged. A subcomputation genuinely rewritten by DCE has no original object to anchor to, and two call paths dropping the same output can still produce duplicate functions under the same cache pattern (fewer than before, but not zero); closing that fully would need structural hashing in the lowering cache key or coherent eviction across the layered DCE caches. The regression tests recreate the stale-outer/evicted-inner cache state deterministically and assert the lowered text matches a fresh lowering, assert no-op DCE returns the identical jaxpr object (both fail before this change), and assert the DCE caches do not keep their key jaxpr alive (guards the sentinel; the naive form of this fix leaked and broke test_cond_memory_leak).
kodlan
force-pushed
the
fix/issue-40312-lowering-trace-history
branch
from
August 30, 2026 18:50
7aa6918 to
a3d6f6b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For #40312 (fixes the reported repro; one narrower subclass remains, see below, so I would keep the issue open rather than auto-closing it).
Lowering the same jitted function could emit different StableHLO depending on what the process traced earlier. The MLIR lowering cache dedups repeated subcomputations by jaxpr object identity, but DCE always rebuilt a new jaxpr even when it eliminated nothing, so that identity was only stable while the right weakref LRU cache entries stayed warm. The caches are layered (the pjit DCE rule has its own cache on top of pe.dce_jaxpr's), and a cache hit only refreshes the outer entry, so unrelated tracing could evict the inner entries while a stale outer entry survived. A later lowering then mixed the stale outer result with freshly recomputed inner ones, got two structurally equal but distinct body jaxprs, and emitted a duplicate function (7 vs 6 funcs in the issue's repro, and a different optimized module on TPU).
The fix: when DCE removes nothing (all outputs and inputs used, no equation changed), pe.dce_jaxpr now returns the input jaxpr itself instead of an equal copy. That anchors object identity at the unchanged leaves, so the lowering dedup for unchanged subjaxprs no longer depends on cache eviction state. Inside the weakref caches the no-op case is stored as a None sentinel rather than the jaxpr, because caching the key as its own value would give the entry a strong reference to its weak key and make it (and everything the jaxpr closes over) immortal; the two caches that re-cache dce results under the same key (the pjit DCE rule's and the closed-call rule's) get the same treatment.
Known residual, out of scope here: a subcomputation that DCE genuinely rewrites has no original object to anchor to, so two call paths that both drop the same output can still produce two structurally equal copies under the same stale-outer/evicted-inner cache pattern (fewer duplicates than before, but not zero). Closing that fully would need structural (content) hashing in the lowering cache key or coherent eviction across the layered DCE caches. The CHANGELOG entry states the fixed scope explicitly.
Tests: a regression test that recreates the stale-outer/evicted-inner cache state deterministically and asserts the lowered text matches a fresh lowering (fails before this change), a test that no-op DCE returns the identical jaxpr object (fails before), and a test that neither DCE cache keeps its key jaxpr alive (guards the sentinel; catches the naive version of this fix, which leaked and broke test_cond_memory_leak).
Not verifiable here: the TPU instruction-count difference from the issue; the function duplication that causes it is platform independent and is fixed for the reported program.