fix(scripts): align agent frontmatter schema with VS Code spec#469
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #469 +/- ##
==========================================
+ Coverage 86.30% 86.53% +0.23%
==========================================
Files 23 23
Lines 4762 4836 +74
==========================================
+ Hits 4110 4185 +75
+ Misses 652 651 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Updates the repo’s agent frontmatter JSON schema to match VS Code Custom Agents spec deltas from #425, and extends the PowerShell schema validator to enforce the newly needed schema constructs during frontmatter linting.
Changes:
- Expanded
agent-frontmatter.schema.jsonwithagents,user-invokable,disable-model-invocation,handoffs.model, and correctedmodelto allow string or string-array. - Enhanced
Test-JsonSchemaValidationto supportoneOfand nestedobject/array.itemsvalidation patterns used by repo schemas. - Added unit tests for
oneOfand nested object/array validation scenarios.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/tests/linting/Validate-MarkdownFrontmatter.Tests.ps1 | Adds unit tests covering oneOf and nested object/array schema validation behavior. |
| scripts/linting/schemas/agent-frontmatter.schema.json | Aligns agent frontmatter schema fields and types with the VS Code spec deltas (while preserving maturity). |
| scripts/linting/Validate-MarkdownFrontmatter.ps1 | Implements validator support for oneOf and nested object/array validation patterns used by the updated schema. |
| .gitignore | Ignores locally generated dependency-pinning artifacts directory. |
| .cspell.json | Adds “invokable” to spelling allowlist. |
katriendg
left a comment
There was a problem hiding this comment.
Thank you so much for picking up an open issue and addressing this @aavetis!
Really appreciate your work on this. Love the code coverage result as well, neat!
I have a few minor comments, if you could please review those? For the rest I have no other comments and we look forward to being able to merge this one in.
Thanks again and hope to see you contribute in the future.
|
@aavetis - once the code comments are resolved, I'll get this approved and merged in. Thanks for the contribution!!! |
|
Thank you for your contribution, @aavetis. We restructured the PR description to follow the project's pull request template. Please review the updated description and check any additional boxes that apply to your changes. |
- Address Copilot review: update validation coverage docs and avoid array += in loops - Fix array conversion to preserve single-item arrays - Add tests to cover oneOf + nested object/array conversion paths (Codecov patch coverage)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Align disable-model-invocation description with microsoft#425/VS Code intent - Aggregate oneOf mismatch errors across subschemas for stable, actionable diagnostics
Exclude IDictionary/hashtable from array type detection to avoid false passes and confusing item errors.
- hoist nested functions to script scope in validator - swap enum/type validation order for correctness - add early exit in oneOf loop after first match - remove contradictory entries from limitations docblock - add test for rejecting non-wildcard agents string 🔧 - Generated by Copilot
2bf78d9 to
7202316
Compare
|
Thank you for this contribution, @aavetis! 🎉 Your work aligning the agent frontmatter schema and validator with the VS Code custom agent spec is a meaningful improvement to the project's schema validation pipeline. We've rebased onto main, addressed the review comments (function hoisting, validation order, oneOf early-exit, docblock cleanup, and the new negative test case), and all 1404 tests pass cleanly. The branch is updated and ready for final review. We really appreciate the time and care you put into this. Great work! |
🤖 I have created a release *beep* *boop* --- ## [3.0.0](hve-core-v2.3.10...hve-core-v3.0.0) (2026-02-20) ### ⚠ BREAKING CHANGES * **skills:** migrate PR reference generation to self-contained skill ([#669](#669)) * restructure RPI collection to HVE Core naming convention ([#668](#668)) ### ✨ Features * **agents:** add agile-coach agent ([#562](#562)) ([de8d86c](de8d86c)) * **agents:** add DT coach agent with tiered instruction loading ([#656](#656)) ([206d3a7](206d3a7)) * **agents:** add product manager advisor and UX/UI designer agents ([#627](#627)) ([539eb8a](539eb8a)) * **agents:** add system architecture reviewer for design trade-offs and ADR creation ([#626](#626)) ([de5cfd6](de5cfd6)) * **build:** pin devcontainer image and align tool parity ([#704](#704)) ([6258b1c](6258b1c)) * **design-thinking:** add manufacturing industry context template ([#682](#682)) ([ce864bf](ce864bf)) * **instructions:** add DT coaching state protocol for session persistence ([#654](#654)) ([5a5be4e](5a5be4e)) * **instructions:** add dt-coaching-identity ambient instruction ([#642](#642)) ([6209a0d](6209a0d)) * **instructions:** add dt-method-01-deep for advanced scope conversation techniques ([#673](#673)) ([cc92ef9](cc92ef9)) * **instructions:** add dt-method-03-deep for advanced input synthesis techniques ([#676](#676)) ([0079a4f](0079a4f)) * **instructions:** add dt-method-09-deep instructions for Method 9 advanced coaching ([#703](#703)) ([150b2a6](150b2a6)) * **instructions:** add dt-method-sequencing ambient instruction ([#650](#650)) ([e465b2f](e465b2f)) * **instructions:** add dt-quality-constraints and design-thinking collection ([#645](#645)) ([17002bd](17002bd)) * **instructions:** add DT-to-RPI handoff contract specification ([#679](#679)) ([87f9962](87f9962)) * **instructions:** add energy industry context template ([#687](#687)) ([41088d8](41088d8)) * **instructions:** add healthcare industry context template ([#686](#686)) ([b2d5281](b2d5281)) * **instructions:** add Method 1 Scope Conversations coaching knowledge ([#651](#651)) ([93e2d48](93e2d48)) * **instructions:** add Method 2 Design Research coaching knowledge ([#652](#652)) ([30f7f3b](30f7f3b)) * **instructions:** add Method 3 Input Synthesis coaching knowledge ([#653](#653)) ([1efdb7d](1efdb7d)) * **instructions:** add Method 7 High-Fidelity Prototypes coaching instruction ([#666](#666)) ([9233eab](9233eab)) * **instructions:** add pull request instructions for PR generation workflow ([#706](#706)) ([73d23eb](73d23eb)) * **instructions:** create DT curriculum content (9 modules) ([#690](#690)) ([9f7378f](9f7378f)), closes [#617](#617) * **instructions:** create dt-method-02-deep.instructions.md ([#700](#700)) ([4d4d0ca](4d4d0ca)) * **instructions:** create dt-method-06-lofi-prototypes.instructions.md ([#684](#684)) ([4d5f757](4d5f757)) * **instructions:** create dt-method-07-deep.instructions.md ([#678](#678)) ([d3ec70d](d3ec70d)) * **instructions:** Create dt-method-08-deep.instructions.md ([#683](#683)) ([d9e1115](d9e1115)) * **instructions:** create dt-method-08-testing.instructions.md ([#681](#681)) ([3008ad8](3008ad8)) * **instructions:** create dt-method-09-iteration.instructions.md ([#685](#685)) ([9d7f4f5](9d7f4f5)) * **instructions:** create dt-rpi-research-context.instructions.md ([#689](#689)) ([34c7b89](34c7b89)) * **instructions:** create manufacturing reference learning scenario ([#692](#692)) ([1bd3994](1bd3994)) * **instructions:** Design Thinking Method 4 brainstorming instruction file ([#664](#664)) ([06f90b0](06f90b0)) * **prompts:** add DT start-project prompt for coaching initialization ([#657](#657)) ([ce583d5](ce583d5)) * **prompts:** add dt-resume-coaching prompt for session recovery ([#665](#665)) ([11b93cb](11b93cb)) * **prompts:** create dt-handoff-problem-space.prompt.md ([#688](#688)) ([277963d](277963d)) * **scripts:** add collection-level maturity field with validation, gating, and notices ([#697](#697)) ([7b1c8e8](7b1c8e8)) * **scripts:** add per-violation CI annotations and colorized console output ([#637](#637)) ([bd7d512](bd7d512)) * **skills:** edit SKILL frontmatter schema, add CI validation, and documentation ([#625](#625)) ([0138a78](0138a78)) * **skills:** mandate unit testing and document language support ([#636](#636)) ([9263617](9263617)) * **skills:** migrate PR reference generation to self-contained skill ([#669](#669)) ([cf8805f](cf8805f)) ### 🐛 Bug Fixes * **collections:** migrate artifacts into collection-based subdirectories ([#658](#658)) ([dfa5261](dfa5261)) * **instructions:** optimize Phase 1 DT token budgets and close [#564](https://github.com/microsoft/hve-core/issues/564)/[#565](https://github.com/microsoft/hve-core/issues/565) gaps ([#675](#675)) ([4f42f00](4f42f00)) * **scripts:** add CI annotations and step summary to copyright header check ([#638](#638)) ([5fa6328](5fa6328)) * **scripts:** add grouped link-lang console diagnostics and failure summary ([#661](#661)) ([4d6871f](4d6871f)) * **scripts:** add per-violation Write-Host and Write-CIAnnotation output to Test-DependencyPinning ([#640](#640)) ([9d3b71d](9d3b71d)) * **scripts:** align agent frontmatter schema with VS Code spec ([#469](#469)) ([254d445](254d445)) * **scripts:** optimize PSScriptAnalyzer linting performance in WSL2 ([#667](#667)) ([f120b93](f120b93)) * **scripts:** stabilize YAML display key ordering in collection manifest ([#701](#701)) ([73c0d2c](73c0d2c)) * **scripts:** use text stubs for plugin links when symlinks unavailable ([#695](#695)) ([d7650a3](d7650a3)) * **skills:** fix powershell test coverage in pr-reference skill ([#699](#699)) ([408e6b7](408e6b7)) ### 📚 Documentation * **dt:** add Method 5 Concepts and Method 6 Lo-Fi Prototypes instructions ([#693](#693)) ([cfdcf11](cfdcf11)) * **hve-guide:** add role-based guides and project lifecycle documentation ([#663](#663)) ([17a85da](17a85da)) ### ♻️ Refactoring * restructure RPI collection to HVE Core naming convention ([#668](#668)) ([120dde0](120dde0)) * **scripts:** consolidate duplicate logging into shared SecurityHelpers module ([#655](#655)) ([627a877](627a877)) * **scripts:** use shared SecurityHelpers and CIHelpers modules in security scripts ([#705](#705)) ([3a0baa7](3a0baa7)) ### 🔧 Maintenance * **deps-dev:** bump markdownlint-cli2 from 0.20.0 to 0.21.0 in the npm-dependencies group ([#609](#609)) ([1486dd7](1486dd7)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: hve-core-release-please[bot] <254602402+hve-core-release-please[bot]@users.noreply.github.com>
Pull Request
Description
Aligns the agent frontmatter JSON schema with the VS Code Custom Agents spec deltas described in #425, and updates the repo's schema validator so the new schema constraints are enforced.
Implementation Notes
Test-JsonSchemaValidationnow supports the schema constructs required for this issue (oneOf, nestedobjectwithrequired/properties, andarray.items)additionalPropertiesis still not enforced by the validator (documented), consistent with prior behaviorIssue #425 Requirements (Checklist)
agent-frontmatter.schema.json:agents,user-invokable,disable-model-invocationmodeltype to allow string OR array of stringshandoffs.promptoptionalhandoffs.model(optional string)maturityfieldRelated Issue(s)
Fixes #425
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.ps1,.sh,.py)Testing
Ran locally:
npm run lint:frontmatternpm run lint:psnpm run test:psnpm run spell-checkpwsh -NoProfile -Command "& scripts/linting/Validate-MarkdownFrontmatter.ps1 -WarningsAsErrors -EnableSchemaValidation -Files (Get-ChildItem -Path .github/agents -Filter *.agent.md -File | Select-Object -ExpandProperty FullName)"Checklist
Required Checks
Required Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:psSecurity Considerations
Additional Notes
dependency-pinning-artifacts/to.gitignoresince it is generated by the dependency pinning scan script during local runs and can otherwise leave the working tree dirty.