Skip to content

Conversation

@Shekharrajak
Copy link

Fix clearing mapped task instances in same group to respect depth-first execution (#40543)

Problem

When clearing a specific mapped task instance within a mapped task group (e.g., etl.e[1]), the system incorrectly cleared ALL downstream task instances (etl.t[] and etl.l[]) instead of only the relevant ones with the same map_index (etl.t[1] and etl.l[1]). This violated the depth-first execution model where each mapped instance should operate independently.

Solution

Added explicit handling in _get_relevant_map_indexes() to detect when two tasks are siblings in the same mapped task group. When ti_count == ancestor_ti_count, the function now returns the same map_index directly, ensuring depth-first execution is maintained.

Changes

  • Modified airflow-core/src/airflow/models/taskinstance.py:

    • Added special case check in _get_relevant_map_indexes() for same-group siblings
    • Returns map_index directly when ti_count == ancestor_ti_count
  • Added tests in airflow-core/tests/unit/models/test_taskinstance.py:

    • test_find_relevant_relatives_downstream_same_group_siblings: Parametrized test for map_indexes 0, 1, 2
    • test_find_relevant_relatives_upstream_same_group_siblings: Test for upstream direction

Testing

All 11 find_relevant_relatives tests pass, including:

  • 6 existing tests (still passing)
  • 4 new tests validating the fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant