TT-15780: add aggregator job to release.yml #7459

sredxny · 2025-10-14T22:28:09Z

User description

Description

Demonstration PR for TT-15780.

This PR shows the expected release.yml after adding the aggregated CI status job via Gromit. The aggregator collects results from all release-related jobs (goreleaser, api-tests, etc.) and produces a single unified status on PRs.

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

I ensured that the documentation is up to date
I explained why this PR updates go.mod in detail with reasoning why it's required
I would like a code coverage CI quality gate exception and have explained why

PR Type

Enhancement, Other

Description

Add aggregated CI status job
Introduce UI tests orchestration
Rename EE to FIPS images/labels
Align workflow to tyk-analytics repo

Diagram Walkthrough

flowchart LR
  GR["goreleaser build & publish"] -- "needs" --> UICTRL["test-controller-ui params"]
  UICTRL -- "outputs" --> UIT["ui-tests matrix run"]
  GR -- "needs" --> APICTRL["test-controller-api params"]
  APICTRL -- "outputs" --> APIT["api-tests matrix run"]
  GR -- "needs" --> DIST["test-controller-distros params"]
  GR -- "needs" --> UDEB["upgrade-deb"]
  GR -- "needs" --> URPM["upgrade-rpm"]
  GR -- "needs" --> RTEST["release-tests"]
  GR -- "needs" --> SBOM["sbom"]
  UIT -- "status" --> AGG["Aggregated CI Status"]
  APIT -- "status" --> AGG
  GR -- "status" --> AGG
  UICTRL & APICTRL & DIST & UDEB & URPM & RTEST -- "always()" --> RLOG["Report GH Logs on Failure"]

File Walkthrough

Relevant files

Enhancement

release.yml `Release workflow: aggregator, UI tests, FIPS/dashboard alignment` .github/workflows/release.yml Add `aggregator-ci-test` to unify PR status. Add UI test controller and `ui-tests` matrix job. Switch repo/image names to `tyk-analytics`, add FIPS image paths/titles. Update upgrade deb/rpm steps for dashboard packages, add failure log reporting.	+207/-57

buger · 2025-10-14T22:28:20Z

💔 The detected issue is not in one of the allowed statuses 💔

Detected Status	In design review	❌
Allowed Statuses	In Dev,In Code Review,Ready for Testing,In Test,In Progress,In Review	✔️

Please ensure your jira story is in one of the allowed statuses

github-actions · 2025-10-14T22:29:20Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Aggregator Robustness The aggregated CI job only checks needs.result != "success". Jobs skipped or cancelled will fail the aggregator; confirm this is desired, and consider handling skipped/cancelled differently or making the needs list conditional based on which suites are enabled. name: Aggregated CI Status runs-on: ubuntu-latest # Dynamically determine which jobs to depend on based on repository configuration needs: [goreleaser, ui-tests, api-tests] if: ${{ always() && github.event_name == 'pull_request' }} steps: - name: Aggregate results run: \| failed=() # Get the needs context as JSON once needs_json='${{ toJSON(needs) }}' # Loop through all jobs in the needs context for job in $(echo "$needs_json" \| jq -r 'keys[]'); do job_result=$(echo "$needs_json" \| jq -r --arg job "$job" '.[$job].result') if [[ "$job_result" != "success" ]]; then failed+=("$job") fi done if (( ${#failed[@]} )); then # Join the failed job names with commas failed_jobs=$(IFS=", "; echo "${failed[]}") echo "❌ Failed jobs ----- : $failed_jobs" exit 1 fi echo "✅ All required jobs succeeded" test-controller-distros: Missing Dependency Guard* Aggregator depends on ui-tests and api-tests unconditionally. If those jobs are excluded on some branches or fail to start, the workflow may fail early. Consider gating the aggregator’s needs dynamically or aligning with test-controller outputs. name: Aggregated CI Status runs-on: ubuntu-latest # Dynamically determine which jobs to depend on based on repository configuration needs: [goreleaser, ui-tests, api-tests] if: ${{ always() && github.event_name == 'pull_request' }} steps: - name: Aggregate results run: \|

github-actions · 2025-10-14T22:29:40Z

API Changes

no api changes detected

github-actions · 2025-10-14T22:30:09Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Ensure jq is available The shell script uses `jq` but the runner may not have it installed by default. Add an explicit installation step to prevent a runtime failure in the aggregator job. .github/workflows/release.yml [499-528] aggregator-ci-test: name: Aggregated CI Status runs-on: ubuntu-latest - # Dynamically determine which jobs to depend on based on repository configuration needs: [goreleaser, ui-tests, api-tests] if: ${{ always() && github.event_name == 'pull_request' }} steps: + - name: Install jq + run: sudo apt-get update && sudo apt-get install -y jq - name: Aggregate results run: \| failed=() - # Get the needs context as JSON once needs_json='${{ toJSON(needs) }}' - ... for job in $(echo "$needs_json" \| jq -r 'keys[]'); do + job_result=$(echo "$needs_json" \| jq -r --arg job "$job" '.[$job].result') + if [[ "$job_result" != "success" ]]; then + failed+=("$job") + fi + done + if (( ${#failed[@]} )); then + failed_jobs=$(IFS=", "; echo "${failed[*]}") + echo "Failed jobs: $failed_jobs" + exit 1 + fi + echo "All required jobs succeeded" Suggestion importance[1-10]: 7 __ Why: The aggregator job uses `jq` without ensuring availability on the runner, which could cause failures. Adding an install step is accurate and improves robustness.	Medium
	Guard missing matrix-dependent outputs The `fips_tags` output is never produced when the `ci_metadata_fips` step is conditional on `matrix.golang_cross == '1.24-bullseye'`. If the matrix expands in the future or this condition is false, downstream jobs reading `needs.goreleaser.outputs.fips_tags` will break. Guard usage sites or ensure the step always runs when outputs are needed. .github/workflows/release.yml [79-82] outputs: - fips_tags: ${{ steps.ci_metadata_fips.outputs.tags }} + fips_tags: ${{ steps.ci_metadata_fips.outputs.tags \|\| '' }} std_tags: ${{ steps.ci_metadata_std.outputs.tags }} commit_author: ${{ steps.set_outputs.outputs.commit_author}} Suggestion importance[1-10]: 6 __ Why: The concern about conditional outputs is valid; guarding `fips_tags` prevents empty outputs from breaking downstream reads. Impact is moderate since current matrix condition likely holds but future changes could break it.	Low
	Align ECR usage with login condition The workflow references `steps.ecr.outputs.registry` while `aws-actions/amazon-ecr-login@v2` only runs when `matrix.golang_cross ==` `'1.24-bullseye'`. Ensure all consumers of `steps.ecr.outputs.registry` are under the same condition to avoid unresolved outputs when the step is skipped. .github/workflows/release.yml [160-171] -- name: Set some outputs for later - id: set_outputs - shell: bash - env: -... - name: Docker metadata for fips CI id: ci_metadata_fips if: ${{ matrix.golang_cross == '1.24-bullseye' }} uses: docker/metadata-action@v5 with: images: \| ${{ steps.ecr.outputs.registry }}/tyk-analytics +- name: Docker metadata for std CI + id: ci_metadata_std + if: ${{ matrix.golang_cross == '1.24-bullseye' }} + uses: docker/metadata-action@v5 + with: + images: \| + ${{ steps.ecr.outputs.registry }}/tyk-analytics Suggestion importance[1-10]: 5 __ Why: The suggestion correctly notes that `steps.ecr` is conditionally set and should be consistently guarded where referenced; however, the existing code already applies the same condition in these places, so impact is limited.	Low

probelabs · 2025-10-14T22:33:27Z

🔍 Code Analysis Results

This PR introduces a new aggregator-ci-test job to the .github/workflows/release.yml file to provide a single, unified status for CI runs. However, the scope of the changes extends far beyond this, including a comprehensive refactoring of the entire release workflow.

Notably, the changes appear to be intended for the tyk-analytics repository, as numerous references to tyk, tyk-gateway, and the current repository's paths are replaced with tyk-analytics and tyk-dashboard. The PR description calls this a "Demonstration PR," which aligns with the significant nature of the changes.

Files Changed Analysis

.github/workflows/release.yml: The only file modified, but with 207 additions and 57 deletions, it represents a complete overhaul of the release pipeline rather than an incremental change.

Architecture & Impact Assessment

What this PR accomplishes:
1. Unified CI Status: Adds an aggregator-ci-test job that consolidates the results of build, UI, and API tests into a single pass/fail status on PRs.
2. Workflow Adaptation: Refactors the workflow to align with the tyk-analytics project, updating repository names, binary names, Docker images, and package paths.
3. Enhanced Testing: Integrates a new UI testing stage (ui-tests) into the CI pipeline.
4. Improved Debugging: Adds a report_logs job to automatically upload logs from failed CI jobs.
Key technical changes introduced:
- A new aggregator-ci-test job that depends on goreleaser, ui-tests, and api-tests.
- New jobs for running UI tests (test-controller-ui, ui-tests).
- Widespread replacement of tyk/tyk-gateway with tyk-analytics/tyk-dashboard.
- The checkout step is modified to use github.event.pull_request.head.sha.
Affected system components:
- The entire CI/CD release pipeline. This change fundamentally alters how pull requests are validated and how releases are built, tested, and published.

Workflow Dependencies:

graph TD
    goreleaser --> test-controller-ui
    goreleaser --> test-controller-api

    test-controller-ui --> ui-tests
    test-controller-api --> api-tests

    subgraph "Aggregated Status Check"
        goreleaser --> aggregator-ci-test
        ui-tests --> aggregator-ci-test
        api-tests --> aggregator-ci-test
    end

    subgraph "Failure Log Reporting"
        test-controller-ui --> report_logs
        test-controller-api --> report_logs
        api-tests --> report_logs
        ui-tests --> report_logs
    end

Scope Discovery & Context Expansion

The changes are confined to CI configuration but have a broad impact on the development lifecycle for this repository. The modifications suggest a move towards a standardized CI process, likely generated by an internal tool (gromit, as mentioned in the file header).
A reviewer should verify that the new package and binary names (tyk-dashboard, tyk-analytics) are correct for this repository's context and that the corresponding UI test suites are in place and configured correctly.

Metadata

Review Effort: 4 / 5
Primary Label: chore

Powered by Visor from Probelabs

Last updated: 2025-10-14T22:33:27.658Z | Triggered by: opened | Commit: 2cfe40a

probelabs · 2025-10-14T22:33:28Z

🔍 Code Analysis Results

✅ Security Check Passed

No security issues found – changes LGTM.

✅ Performance Check Passed

No performance issues found – changes LGTM.

✅ Quality Check Passed

No quality issues found – changes LGTM.

Style Issues (1)

Severity	Location	Issue
🟢 Info	`.github/workflows/release.yml:401`	The comment `# Dynamically determine which jobs to depend on based on repository configuration` is misleading as the `needs` list that follows is statically defined as `[goreleaser, api-tests]`. To avoid confusion, consider updating the comment to reflect the current static nature of the dependencies or removing it if there are no immediate plans for dynamic determination. 💡 Suggestion Update the comment to accurately describe the purpose of the job, for example: `# Aggregate results from key CI jobs for a unified PR status.`

✅ Dependency Check Passed

No dependency issues found – changes LGTM.

✅ Connectivity Check Passed

No connectivity issues found – changes LGTM.

Powered by Visor from Probelabs

Last updated: 2025-10-14T22:34:33.411Z | Triggered by: synchronize | Commit: 3984d35

sonarqubecloud · 2025-10-14T22:54:17Z

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

TT-15780: add aggregator job to release.yml

2cfe40a

sredxny requested a review from a team as a code owner October 14, 2025 22:28

buger added Story Tyk Product Development labels Oct 14, 2025

github-actions bot added the Review effort 3/5 label Oct 14, 2025

set correct release.yml file

3984d35

github-actions bot added chore review/effort:4 labels Oct 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

TT-15780: add aggregator job to release.yml #7459

TT-15780: add aggregator job to release.yml #7459

Uh oh!

sredxny commented Oct 14, 2025 •

edited by github-actions bot

Loading

buger commented Oct 14, 2025

github-actions bot commented Oct 14, 2025

github-actions bot commented Oct 14, 2025

github-actions bot commented Oct 14, 2025

probelabs bot commented Oct 14, 2025

probelabs bot commented Oct 14, 2025 •

edited

Loading

sonarqubecloud bot commented Oct 14, 2025

Labels

3 participants

TT-15780: add aggregator job to release.yml #7459

Are you sure you want to change the base?

TT-15780: add aggregator job to release.yml #7459

Uh oh!

Conversation

sredxny commented Oct 14, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Checklist

PR Type

Description

Diagram Walkthrough

File Walkthrough

buger commented Oct 14, 2025

github-actions bot commented Oct 14, 2025

PR Reviewer Guide 🔍

github-actions bot commented Oct 14, 2025

github-actions bot commented Oct 14, 2025

PR Code Suggestions ✨

probelabs bot commented Oct 14, 2025

🔍 Code Analysis Results

Files Changed Analysis

Architecture & Impact Assessment

Scope Discovery & Context Expansion

probelabs bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Code Analysis Results

✅ Security Check Passed

✅ Performance Check Passed

✅ Quality Check Passed

Style Issues (1)

✅ Dependency Check Passed

✅ Connectivity Check Passed

sonarqubecloud bot commented Oct 14, 2025

Quality Gate passed

Labels

3 participants

sredxny commented Oct 14, 2025 •

edited by github-actions bot

Loading

probelabs bot commented Oct 14, 2025 •

edited

Loading