Skip to content

fix(docs): contributor guide describes a PR test step gated to manual dispatch #198

Open

Description

@coygeek

Summary

The contributor guide describes make test as the suite used by pull-request CI, while the deterministic Python test step in the CI workflow is restricted to manual dispatch. This gives contributors conflicting information about when that suite runs. This report asks for an accurate description of the existing CI policy; it does not assume that maintainers intend to run the suite on every pull request.

Steps to reproduce

  1. Open the checkout identified under Runtime or environment.
  2. Read the Test layers paragraph in CONTRIBUTING.md:22-24.
  3. Read the triggers in .github/workflows/ci.yml:3-6 and the deterministic Python test step at lines 66-78.
  4. Compare the guide's pull-request CI claim with the step condition for a pull_request event.

Expected behavior

The contributor guide should accurately state which CI events run the deterministic Python suite, consistently with the workflow's configured conditions.

Actual behavior

The guide states, "This is the same suite used by pull-request CI." The workflow declares a pull_request trigger, but the Run deterministic Python tests step has if: github.event_name == 'workflow_dispatch'. That condition excludes a pull_request event. The mismatch was verified by reading both files in a fresh checkout. No CI run was triggered and the Python test suite was not executed for this report.

Evidence

  • Expected source: The contributor guide's Test layers paragraph linked below describes the deterministic suite as used by pull-request CI.
  • Failure source: The CI workflow linked below guards the deterministic Python test step with an equality comparison to the manual-dispatch event name, excluding pull-request events.
  • Evidence provenance: static
  • Local verification: static-proof
  • Reproduction completeness: complete

both observations come from the same public source checkout. runtime reproduction was not run. The documentation is directly readable, and the workflow declares both event types while placing the deterministic suite behind a condition that is false for a pull-request event.

Contributor guide, lines 20-24, workflow event triggers, lines 3-6, and deterministic Python step, lines 64-78 show the documented claim and configured restriction.

Restoration check

Read the guide and workflow together in the same revision. The guide's statement about pull-request execution of the deterministic Python suite should match the effective workflow conditions. Preserve the distinction between this suite and the separate frontend tests and package smoke tests; this report does not claim that pull requests run no tests at all. The intended CI policy remains the maintainers' choice.

Runtime or environment

Source-only inspection of the public main branch at commit 351ca8422f7b5b54e80a9c1ce03a222e02415b6b. The repository's GitHub releases and tags endpoints returned no entries. No Artemis runtime, platform behavior, or live CI execution was measured.

References

Open issue #18 already discusses the manual-only test gate and other CI/test failures. This report has a separate closure criterion: the contributor guide must accurately describe whichever event policy maintainers intend. Updating that wording can close this documentation issue without resolving the runtime or test failures in the related issue.

Triage assessment

  • Impact level: P3
  • Assessment status: supported
  • Impact basis: This is documentation drift about CI coverage; no additional runtime regression or test failure is asserted by this report.
  • Workaround status: available
  • Workaround basis: Contributors can inspect the workflow condition to see that this deterministic Python step is limited to manual dispatch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions