Skip to content

fix(scripts): align agent frontmatter schema with VS Code spec#469

Merged
WilliamBerryiii merged 11 commits intomicrosoft:mainfrom
aavetis:codex/issue-425-agent-frontmatter-schema-only
Feb 20, 2026
Merged

fix(scripts): align agent frontmatter schema with VS Code spec#469
WilliamBerryiii merged 11 commits intomicrosoft:mainfrom
aavetis:codex/issue-425-agent-frontmatter-schema-only

Conversation

@aavetis
Copy link
Contributor

@aavetis aavetis commented Feb 10, 2026

Pull Request

Description

Aligns the agent frontmatter JSON schema with the VS Code Custom Agents spec deltas described in #425, and updates the repo's schema validator so the new schema constraints are enforced.

Implementation Notes

  • Test-JsonSchemaValidation now supports the schema constructs required for this issue (oneOf, nested object with required/properties, and array.items)
  • additionalProperties is still not enforced by the validator (documented), consistent with prior behavior

Issue #425 Requirements (Checklist)

  • Add missing fields to agent-frontmatter.schema.json: agents, user-invokable, disable-model-invocation
  • Fix model type to allow string OR array of strings
  • Make handoffs.prompt optional
  • Add handoffs.model (optional string)
  • Preserve HVE-Core specific maturity field

Related Issue(s)

Fixes #425

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

Ran locally:

  • npm run lint:frontmatter
  • npm run lint:ps
  • npm run test:ps
  • npm run spell-check
  • Validated all agents with schema validation enabled:
    • pwsh -NoProfile -Command "& scripts/linting/Validate-MarkdownFrontmatter.ps1 -WarningsAsErrors -EnableSchemaValidation -Files (Get-ChildItem -Path .github/agents -Filter *.agent.md -File | Select-Object -ExpandProperty FullName)"

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

  • Adds dependency-pinning-artifacts/ to .gitignore since it is generated by the dependency pinning scan script during local runs and can otherwise leave the working tree dirty.
Copilot AI review requested due to automatic review settings February 10, 2026 17:00
@aavetis aavetis requested a review from a team as a code owner February 10, 2026 17:00
@codecov-commenter
Copy link

codecov-commenter commented Feb 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.53%. Comparing base (73c0d2c) to head (18700ad).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #469      +/-   ##
==========================================
+ Coverage   86.30%   86.53%   +0.23%     
==========================================
  Files          23       23              
  Lines        4762     4836      +74     
==========================================
+ Hits         4110     4185      +75     
+ Misses        652      651       -1     
Flag Coverage Δ
pester 86.53% <100.00%> (+0.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/linting/Validate-MarkdownFrontmatter.ps1 74.66% <100.00%> (+8.89%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the repo’s agent frontmatter JSON schema to match VS Code Custom Agents spec deltas from #425, and extends the PowerShell schema validator to enforce the newly needed schema constructs during frontmatter linting.

Changes:

  • Expanded agent-frontmatter.schema.json with agents, user-invokable, disable-model-invocation, handoffs.model, and corrected model to allow string or string-array.
  • Enhanced Test-JsonSchemaValidation to support oneOf and nested object/array.items validation patterns used by repo schemas.
  • Added unit tests for oneOf and nested object/array validation scenarios.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/tests/linting/Validate-MarkdownFrontmatter.Tests.ps1 Adds unit tests covering oneOf and nested object/array schema validation behavior.
scripts/linting/schemas/agent-frontmatter.schema.json Aligns agent frontmatter schema fields and types with the VS Code spec deltas (while preserving maturity).
scripts/linting/Validate-MarkdownFrontmatter.ps1 Implements validator support for oneOf and nested object/array validation patterns used by the updated schema.
.gitignore Ignores locally generated dependency-pinning artifacts directory.
.cspell.json Adds “invokable” to spelling allowlist.
@aavetis aavetis marked this pull request as draft February 10, 2026 17:16
@aavetis aavetis marked this pull request as ready for review February 10, 2026 17:35
Copilot AI review requested due to automatic review settings February 10, 2026 17:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.

Copilot AI review requested due to automatic review settings February 10, 2026 19:02
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Copy link
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for picking up an open issue and addressing this @aavetis!

Really appreciate your work on this. Love the code coverage result as well, neat!
I have a few minor comments, if you could please review those? For the rest I have no other comments and we look forward to being able to merge this one in.

Thanks again and hope to see you contribute in the future.

@WilliamBerryiii
Copy link
Member

@aavetis - once the code comments are resolved, I'll get this approved and merged in. Thanks for the contribution!!!

@katriendg katriendg added this to the v2.4.0 milestone Feb 12, 2026
Copilot AI review requested due to automatic review settings February 13, 2026 01:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

@WilliamBerryiii
Copy link
Member

Thank you for your contribution, @aavetis. We restructured the PR description to follow the project's pull request template. Please review the updated description and check any additional boxes that apply to your changes.

aavetis and others added 9 commits February 19, 2026 20:35
- Address Copilot review: update validation coverage docs and avoid array += in loops

- Fix array conversion to preserve single-item arrays

- Add tests to cover oneOf + nested object/array conversion paths (Codecov patch coverage)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Align disable-model-invocation description with microsoft#425/VS Code intent

- Aggregate oneOf mismatch errors across subschemas for stable, actionable diagnostics
Exclude IDictionary/hashtable from array type detection to avoid false passes and confusing item errors.
- hoist nested functions to script scope in validator
- swap enum/type validation order for correctness
- add early exit in oneOf loop after first match
- remove contradictory entries from limitations docblock
- add test for rejecting non-wildcard agents string

🔧 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii force-pushed the codex/issue-425-agent-frontmatter-schema-only branch from 2bf78d9 to 7202316 Compare February 20, 2026 04:54
@WilliamBerryiii
Copy link
Member

Thank you for this contribution, @aavetis! 🎉 Your work aligning the agent frontmatter schema and validator with the VS Code custom agent spec is a meaningful improvement to the project's schema validation pipeline.

We've rebased onto main, addressed the review comments (function hoisting, validation order, oneOf early-exit, docblock cleanup, and the new negative test case), and all 1404 tests pass cleanly. The branch is updated and ready for final review.

We really appreciate the time and care you put into this. Great work!

@WilliamBerryiii WilliamBerryiii merged commit 254d445 into microsoft:main Feb 20, 2026
19 checks passed
WilliamBerryiii pushed a commit that referenced this pull request Feb 20, 2026
🤖 I have created a release *beep* *boop*
---


##
[3.0.0](hve-core-v2.3.10...hve-core-v3.0.0)
(2026-02-20)


### ⚠ BREAKING CHANGES

* **skills:** migrate PR reference generation to self-contained skill
([#669](#669))
* restructure RPI collection to HVE Core naming convention
([#668](#668))

### ✨ Features

* **agents:** add agile-coach agent
([#562](#562))
([de8d86c](de8d86c))
* **agents:** add DT coach agent with tiered instruction loading
([#656](#656))
([206d3a7](206d3a7))
* **agents:** add product manager advisor and UX/UI designer agents
([#627](#627))
([539eb8a](539eb8a))
* **agents:** add system architecture reviewer for design trade-offs and
ADR creation ([#626](#626))
([de5cfd6](de5cfd6))
* **build:** pin devcontainer image and align tool parity
([#704](#704))
([6258b1c](6258b1c))
* **design-thinking:** add manufacturing industry context template
([#682](#682))
([ce864bf](ce864bf))
* **instructions:** add DT coaching state protocol for session
persistence ([#654](#654))
([5a5be4e](5a5be4e))
* **instructions:** add dt-coaching-identity ambient instruction
([#642](#642))
([6209a0d](6209a0d))
* **instructions:** add dt-method-01-deep for advanced scope
conversation techniques
([#673](#673))
([cc92ef9](cc92ef9))
* **instructions:** add dt-method-03-deep for advanced input synthesis
techniques ([#676](#676))
([0079a4f](0079a4f))
* **instructions:** add dt-method-09-deep instructions for Method 9
advanced coaching
([#703](#703))
([150b2a6](150b2a6))
* **instructions:** add dt-method-sequencing ambient instruction
([#650](#650))
([e465b2f](e465b2f))
* **instructions:** add dt-quality-constraints and design-thinking
collection ([#645](#645))
([17002bd](17002bd))
* **instructions:** add DT-to-RPI handoff contract specification
([#679](#679))
([87f9962](87f9962))
* **instructions:** add energy industry context template
([#687](#687))
([41088d8](41088d8))
* **instructions:** add healthcare industry context template
([#686](#686))
([b2d5281](b2d5281))
* **instructions:** add Method 1 Scope Conversations coaching knowledge
([#651](#651))
([93e2d48](93e2d48))
* **instructions:** add Method 2 Design Research coaching knowledge
([#652](#652))
([30f7f3b](30f7f3b))
* **instructions:** add Method 3 Input Synthesis coaching knowledge
([#653](#653))
([1efdb7d](1efdb7d))
* **instructions:** add Method 7 High-Fidelity Prototypes coaching
instruction ([#666](#666))
([9233eab](9233eab))
* **instructions:** add pull request instructions for PR generation
workflow ([#706](#706))
([73d23eb](73d23eb))
* **instructions:** create DT curriculum content (9 modules)
([#690](#690))
([9f7378f](9f7378f)),
closes [#617](#617)
* **instructions:** create dt-method-02-deep.instructions.md
([#700](#700))
([4d4d0ca](4d4d0ca))
* **instructions:** create dt-method-06-lofi-prototypes.instructions.md
([#684](#684))
([4d5f757](4d5f757))
* **instructions:** create dt-method-07-deep.instructions.md
([#678](#678))
([d3ec70d](d3ec70d))
* **instructions:** Create dt-method-08-deep.instructions.md
([#683](#683))
([d9e1115](d9e1115))
* **instructions:** create dt-method-08-testing.instructions.md
([#681](#681))
([3008ad8](3008ad8))
* **instructions:** create dt-method-09-iteration.instructions.md
([#685](#685))
([9d7f4f5](9d7f4f5))
* **instructions:** create dt-rpi-research-context.instructions.md
([#689](#689))
([34c7b89](34c7b89))
* **instructions:** create manufacturing reference learning scenario
([#692](#692))
([1bd3994](1bd3994))
* **instructions:** Design Thinking Method 4 brainstorming instruction
file ([#664](#664))
([06f90b0](06f90b0))
* **prompts:** add DT start-project prompt for coaching initialization
([#657](#657))
([ce583d5](ce583d5))
* **prompts:** add dt-resume-coaching prompt for session recovery
([#665](#665))
([11b93cb](11b93cb))
* **prompts:** create dt-handoff-problem-space.prompt.md
([#688](#688))
([277963d](277963d))
* **scripts:** add collection-level maturity field with validation,
gating, and notices
([#697](#697))
([7b1c8e8](7b1c8e8))
* **scripts:** add per-violation CI annotations and colorized console
output ([#637](#637))
([bd7d512](bd7d512))
* **skills:** edit SKILL frontmatter schema, add CI validation, and
documentation ([#625](#625))
([0138a78](0138a78))
* **skills:** mandate unit testing and document language support
([#636](#636))
([9263617](9263617))
* **skills:** migrate PR reference generation to self-contained skill
([#669](#669))
([cf8805f](cf8805f))


### 🐛 Bug Fixes

* **collections:** migrate artifacts into collection-based
subdirectories
([#658](#658))
([dfa5261](dfa5261))
* **instructions:** optimize Phase 1 DT token budgets and close
[#564](https://github.com/microsoft/hve-core/issues/564)/[#565](https://github.com/microsoft/hve-core/issues/565)
gaps ([#675](#675))
([4f42f00](4f42f00))
* **scripts:** add CI annotations and step summary to copyright header
check ([#638](#638))
([5fa6328](5fa6328))
* **scripts:** add grouped link-lang console diagnostics and failure
summary ([#661](#661))
([4d6871f](4d6871f))
* **scripts:** add per-violation Write-Host and Write-CIAnnotation
output to Test-DependencyPinning
([#640](#640))
([9d3b71d](9d3b71d))
* **scripts:** align agent frontmatter schema with VS Code spec
([#469](#469))
([254d445](254d445))
* **scripts:** optimize PSScriptAnalyzer linting performance in WSL2
([#667](#667))
([f120b93](f120b93))
* **scripts:** stabilize YAML display key ordering in collection
manifest ([#701](#701))
([73c0d2c](73c0d2c))
* **scripts:** use text stubs for plugin links when symlinks unavailable
([#695](#695))
([d7650a3](d7650a3))
* **skills:** fix powershell test coverage in pr-reference skill
([#699](#699))
([408e6b7](408e6b7))


### 📚 Documentation

* **dt:** add Method 5 Concepts and Method 6 Lo-Fi Prototypes
instructions ([#693](#693))
([cfdcf11](cfdcf11))
* **hve-guide:** add role-based guides and project lifecycle
documentation ([#663](#663))
([17a85da](17a85da))


### ♻️ Refactoring

* restructure RPI collection to HVE Core naming convention
([#668](#668))
([120dde0](120dde0))
* **scripts:** consolidate duplicate logging into shared SecurityHelpers
module ([#655](#655))
([627a877](627a877))
* **scripts:** use shared SecurityHelpers and CIHelpers modules in
security scripts
([#705](#705))
([3a0baa7](3a0baa7))


### 🔧 Maintenance

* **deps-dev:** bump markdownlint-cli2 from 0.20.0 to 0.21.0 in the
npm-dependencies group
([#609](#609))
([1486dd7](1486dd7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: hve-core-release-please[bot] <254602402+hve-core-release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants