Skip to content

feat(scripts): add Write-CIStepSummary markdown table to Test-SHAStaleness github output#660

Merged
katriendg merged 14 commits intomicrosoft:mainfrom
AhmedMustafa249:bug/add-missing-Write-CIStepSummary-in-Test-SHAStaleness.ps1
Feb 23, 2026
Merged

feat(scripts): add Write-CIStepSummary markdown table to Test-SHAStaleness github output#660
katriendg merged 14 commits intomicrosoft:mainfrom
AhmedMustafa249:bug/add-missing-Write-CIStepSummary-in-Test-SHAStaleness.ps1

Conversation

@AhmedMustafa249
Copy link
Contributor

@AhmedMustafa249 AhmedMustafa249 commented Feb 18, 2026

Pull Request

Description

  • build markdown summary table with dependency name, SHA age, threshold, and stale/current status in github output branch of Write-OutputResult
  • invoke Write-CIStepSummary after per-dependency Write-CIAnnotation loop
  • add 12 Pester tests asserting Write-CIAnnotation call counts and Write-CIStepSummary content

Related Issue(s)

Fixes #633

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

  • 12 new Pester tests added across two contexts (GitHub output format with stale dependencies and GitHub output format with no stale dependencies)
  • Tests verify Write-CIAnnotation call counts (per-item Warning, aggregate Error/Notice) and Write-CIStepSummary content (table header, dependency names, stale status, totals, all-clear message)
  • Mocks placed inside each It block to work around Pester 5 module-scoping behavior with module-imported functions
  • npm run lint:ps passes clean
  • npm run test:ps passes

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

Minor findings:

  • Mixed stale/current test gap: Both test dependencies use DaysOld > MaxAge (45 and 90 vs threshold 30). No test exercises the ✅ Current row status alongside a stale row in the same table. A mixed-scenario test (e.g., DaysOld = 10 + DaysOld = 45) would validate the stale/clean counting logic and both status indicator branches.

  • Message content not asserted: Tests verify -Level parameter filters but not -Message content (e.g., severity tag [Low], dependency details). Adding -Message ParameterFilter assertions would strengthen coverage.

Potential Follow-up work would include:

  • Adding a mixed stale/current test context with dependencies both above and below $MaxAge
  • Add -Message content assertions for Write-CIAnnotation calls
…eness github output

- build markdown summary table with dependency name, SHA age, threshold, and stale/current status in github output branch of Write-OutputResult
- invoke Write-CIStepSummary after per-dependency Write-CIAnnotation loop
- add 12 Pester tests asserting Write-CIAnnotation call counts and Write-CIStepSummary content
@AhmedMustafa249 AhmedMustafa249 marked this pull request as ready for review February 18, 2026 09:06
@AhmedMustafa249 AhmedMustafa249 requested a review from a team as a code owner February 18, 2026 09:06
@katriendg katriendg requested review from Copilot February 18, 2026 12:46
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds GitHub Actions step summary support to the Test-SHAStaleness.ps1 script by implementing Write-CIStepSummary with a markdown table showing dependency staleness results. The change addresses issue #633 where SHA staleness checks produced CI annotations but left the job summary tab empty.

Changes:

  • Add markdown table generation in the GitHub output format branch of Write-OutputResult showing dependency name, SHA age, threshold, and status
  • Call Write-CIStepSummary after per-dependency annotations to populate the GitHub Actions job summary
  • Add 12 Pester tests across two contexts to verify annotation counts and summary content

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
scripts/security/Test-SHAStaleness.ps1 Adds Write-CIStepSummary call with markdown table for GitHub output format; adds UTF-8 BOM to first line
scripts/tests/security/Test-SHAStaleness.Tests.ps1 Adds two new test contexts with 12 test cases mocking Write-CIAnnotation and Write-CIStepSummary
@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.86%. Comparing base (ac939aa) to head (3e0bbb8).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #660      +/-   ##
==========================================
+ Coverage   84.51%   84.86%   +0.35%     
==========================================
  Files          24       24              
  Lines        4868     4877       +9     
==========================================
+ Hits         4114     4139      +25     
+ Misses        754      738      -16     
Flag Coverage Δ
pester 84.86% <100.00%> (+0.35%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/security/Test-SHAStaleness.ps1 69.15% <100.00%> (+4.26%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
AhmedMustafa249 and others added 4 commits February 18, 2026 17:34
…ged table summary to use "Found" instead of "Scanned" 3) set $status to "Stale" by default
…inputs

- remove mixed stale/current assumptions in summary and tests
- update Pester assertions to match stale-only dependency behavior
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @AhmedMustafa249 for your contribution!
We have a few open comments to address before we can merge this one. Should be minor hopefully.

@AhmedMustafa249
Copy link
Contributor Author

Thank you @AhmedMustafa249 for your contribution! We have a few open comments to address before we can merge this one. Should be minor hopefully.

Hey @katriendg, thank you for all the comments!🙏 I'll get to resolving everything so we can get this merged asap 🫡

…redundant $staleCount variable

- Consolidate repeated Mock/Write-OutputResult calls into BeforeEach blocks in the GitHub output format test contexts
- Remove the $staleCount variable that was always identical to $totalCount in Write-OutputResult.
- Resave scripts/security/Test-SHAStaleness.ps1 as UTF-8
Copy link
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this looks good now, thank you for your contribution!

@katriendg
Copy link
Contributor

@AhmedMustafa249 Can you please run the powershell linting and fix (two references), and see if the Powershell tests was a transient issue or you also get the issue and then fix? Thanks.

- Fixed two linting references caused by unsupported emojis in UTF-8 file and overlook em dash
- Replaced 'Write-OutputResult' with 'Write-SecurityOutput' due to refactoring done by PR microsoft#705
@AhmedMustafa249
Copy link
Contributor Author

@katriendg All good now! 😄

@katriendg katriendg merged commit cd3334a into microsoft:main Feb 23, 2026
19 checks passed
WilliamBerryiii pushed a commit that referenced this pull request Feb 28, 2026
## Pre-Release 3.1.44

### ✨ Features

- add Docusaurus 3 documentation site with GitHub Pages deployment
(#680)
- add workflow permissions validation for OpenSSF Scorecard compliance
(#759)
- add DT coach return path handoff to task-researcher (#591) (#758)
- add DT subagent handoff workflow instructions (#592) (#757)
- create dt-method-06-deep.instructions.md (#602) (#748)
- create dt-method-05-deep.instructions.md (#747)
- add DT-aware task-implementor context instructions (#755)
- extract embedded PowerShell from workflows into testable scripts
(#738)
- add gitleaks binary-based secret scanning as PR gate (#734)
- add SBOM generation, attestation, and diff tooling to release pipeline
(#730)
- add dt-learning-tutor agent for DT education (#662)
- add DT image prompt generation guidance for Method 5 (#726)
- add DT-aware task-reviewer review context (#714)
- add dt-method-next routing prompt (#713)
- create dt-method-04-deep.instructions.md (#709)
- add Implementation Space exit handoff prompt for DT workflows (#708)
- add Write-CIStepSummary markdown table to Test-SHAStaleness github
output (#660)
- add dt-handoff-solution-space prompt for Solution Spac… (#707)

### 🐛 Bug Fixes

- update sidebar link color to meet WCAG AA contrast requirements (#814)
- harden even/odd versioning against regression and syntax errors (#816)
- replace even/odd versioning with SemVer -rc.N suffixes (#811)
- ensure prerelease label exists before PR creation (#806)
- replace Docusaurus favicons with Microsoft logo (#808)
- add missing subagents and shared instructions to collection manifests
(#804)
- standardize file path conventions for copilot-tracking output (#784)
- enforce project-scoped artifact isolation across DT files (#766)
- add top-level permissions to copilot-setup-steps.yml (#760)
- update broken file directives and markdown links after collection
directory reorg (#743)
- add pre-release companion pipeline with even/odd versioning (#735)
- exclude auto-generated CHANGELOG.md from spell check (#756)
- add job-level permissions to extension-publish.yml (#729)
- resolve handoff dependencies using display names (#727)
- add job-level permissions to validate-version in
extension-publish-prerelease (#731)
- replace parent-directory VS Code settings paths with per-subdirectory
enumeration (#732)

### 📚 Documentation

- add Design Thinking documentation and DT-to-RPI handoff (#789)
- add customization guides for HVE Core artifacts (#772)
- reconcile documentation against implementation (#771)
- document accepted Token-Permissions risks and add
lint:dependency-pinning (#763)
- add Design Thinking section to hve-core-all collection description
(#762)

### ♻️ Refactoring

- move collection scripts from plugins to collections (#728)
- remove duplicate git diff logic in frontmatter validator (#473)

### 🔧 Maintenance

- bump basic-ftp from 5.0.5 to 5.2.0 (#780)
- standardize script path references in SKILL.md files (#768)
- bump the github-actions group across 1 directory with 2 updates (#752)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
WilliamBerryiii pushed a commit that referenced this pull request Feb 28, 2026
## Pre-Release 3.1.46

### ✨ Features

- add Docusaurus 3 documentation site with GitHub Pages deployment
(#680)
- add workflow permissions validation for OpenSSF Scorecard compliance
(#759)
- add DT coach return path handoff to task-researcher (#591) (#758)
- add DT subagent handoff workflow instructions (#592) (#757)
- create dt-method-06-deep.instructions.md (#602) (#748)
- create dt-method-05-deep.instructions.md (#747)
- add DT-aware task-implementor context instructions (#755)
- extract embedded PowerShell from workflows into testable scripts
(#738)
- add gitleaks binary-based secret scanning as PR gate (#734)
- add SBOM generation, attestation, and diff tooling to release pipeline
(#730)
- add dt-learning-tutor agent for DT education (#662)
- add DT image prompt generation guidance for Method 5 (#726)
- add DT-aware task-reviewer review context (#714)
- add dt-method-next routing prompt (#713)
- create dt-method-04-deep.instructions.md (#709)
- add Implementation Space exit handoff prompt for DT workflows (#708)
- add Write-CIStepSummary markdown table to Test-SHAStaleness github
output (#660)
- add dt-handoff-solution-space prompt for Solution Spac… (#707)

### 🐛 Bug Fixes

- update prerelease publish to use even/odd convention (#822)
- update sidebar link color to meet WCAG AA contrast requirements (#814)
- harden even/odd versioning against regression and syntax errors (#816)
- replace even/odd versioning with SemVer -rc.N suffixes (#811)
- ensure prerelease label exists before PR creation (#806)
- replace Docusaurus favicons with Microsoft logo (#808)
- add missing subagents and shared instructions to collection manifests
(#804)
- standardize file path conventions for copilot-tracking output (#784)
- enforce project-scoped artifact isolation across DT files (#766)
- add top-level permissions to copilot-setup-steps.yml (#760)
- update broken file directives and markdown links after collection
directory reorg (#743)
- add pre-release companion pipeline with even/odd versioning (#735)
- exclude auto-generated CHANGELOG.md from spell check (#756)
- add job-level permissions to extension-publish.yml (#729)
- resolve handoff dependencies using display names (#727)
- add job-level permissions to validate-version in
extension-publish-prerelease (#731)
- replace parent-directory VS Code settings paths with per-subdirectory
enumeration (#732)

### 📚 Documentation

- add Design Thinking documentation and DT-to-RPI handoff (#789)
- add customization guides for HVE Core artifacts (#772)
- reconcile documentation against implementation (#771)
- document accepted Token-Permissions risks and add
lint:dependency-pinning (#763)
- add Design Thinking section to hve-core-all collection description
(#762)

### ♻️ Refactoring

- move collection scripts from plugins to collections (#728)
- remove duplicate git diff logic in frontmatter validator (#473)

### 🔧 Maintenance

- pre-release 3.1.44 (#819)
- bump basic-ftp from 5.0.5 to 5.2.0 (#780)
- standardize script path references in SKILL.md files (#768)
- bump the github-actions group across 1 directory with 2 updates (#752)

---
*Managed automatically by pre-release workflow.*

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants