Repository navigation
Description
Summary
The contributor guide describes make test as the suite used by pull-request CI, while the deterministic Python test step in the CI workflow is restricted to manual dispatch. This gives contributors conflicting information about when that suite runs. This report asks for an accurate description of the existing CI policy; it does not assume that maintainers intend to run the suite on every pull request.
Steps to reproduce
- Open the checkout identified under Runtime or environment.
- Read the Test layers paragraph in
CONTRIBUTING.md:22-24. - Read the triggers in
.github/workflows/ci.yml:3-6and the deterministic Python test step at lines 66-78. - Compare the guide's pull-request CI claim with the step condition for a
pull_requestevent.
Expected behavior
The contributor guide should accurately state which CI events run the deterministic Python suite, consistently with the workflow's configured conditions.
Actual behavior
The guide states, "This is the same suite used by pull-request CI." The workflow declares a pull_request trigger, but the Run deterministic Python tests step has if: github.event_name == 'workflow_dispatch'. That condition excludes a pull_request event. The mismatch was verified by reading both files in a fresh checkout. No CI run was triggered and the Python test suite was not executed for this report.
Evidence
- Expected source: The contributor guide's Test layers paragraph linked below describes the deterministic suite as used by pull-request CI.
- Failure source: The CI workflow linked below guards the deterministic Python test step with an equality comparison to the manual-dispatch event name, excluding pull-request events.
- Evidence provenance: static
- Local verification: static-proof
- Reproduction completeness: complete
both observations come from the same public source checkout. runtime reproduction was not run. The documentation is directly readable, and the workflow declares both event types while placing the deterministic suite behind a condition that is false for a pull-request event.
Contributor guide, lines 20-24, workflow event triggers, lines 3-6, and deterministic Python step, lines 64-78 show the documented claim and configured restriction.
Restoration check
Read the guide and workflow together in the same revision. The guide's statement about pull-request execution of the deterministic Python suite should match the effective workflow conditions. Preserve the distinction between this suite and the separate frontend tests and package smoke tests; this report does not claim that pull requests run no tests at all. The intended CI policy remains the maintainers' choice.
Runtime or environment
Source-only inspection of the public main branch at commit 351ca8422f7b5b54e80a9c1ce03a222e02415b6b. The repository's GitHub releases and tags endpoints returned no entries. No Artemis runtime, platform behavior, or live CI execution was measured.
References
Open issue #18 already discusses the manual-only test gate and other CI/test failures. This report has a separate closure criterion: the contributor guide must accurately describe whichever event policy maintainers intend. Updating that wording can close this documentation issue without resolving the runtime or test failures in the related issue.
Triage assessment
- Impact level: P3
- Assessment status: supported
- Impact basis: This is documentation drift about CI coverage; no additional runtime regression or test failure is asserted by this report.
- Workaround status: available
- Workaround basis: Contributors can inspect the workflow condition to see that this deterministic Python step is limited to manual dispatch.
Activity