Skip to content
This repository was archived by the owner on Sep 4, 2025. It is now read-only.

Commit b08d6b5

Browse files
authored
Minor upgrades and updated docs for ToolDescriptionEvaluator (#1016)
* Updated special character handling * Removed unnecessary log messages * Updated docs and env variable handling * Re-ran the utility * Applied PR feedback and updated space replacement character * Re-ran the utility * Updated single tool validation logic * Fixed escaping issues and updated results * Added a new build task for ToolDescriptionEvaluator * Applied PR feedback
1 parent f2de3a4 commit b08d6b5

File tree

11 files changed

+11317
-9774
lines changed

11 files changed

+11317
-9774
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
- [ ] Updated `README.md` documentation
2121
- [ ] Updated command list in `/docs/azmcp-commands.md`
2222
- [ ] Updated test prompts in `/docs/e2eTestPrompts.md`
23-
- [ ] For new or modified tool descriptions, ran the `eng/tools/ToolDescriptionEvaluator` tool and obtained a result >= 0.4
23+
- [ ] For new or modified tool descriptions, ran [`ToolDescriptionEvaluator`](https://github.com/Azure/azure-mcp/blob/main/eng/tools/ToolDescriptionEvaluator/Quickstart.md) and obtained a score of `0.4` or more and a top 3 ranking for all related test prompts
2424
- [ ] 👉 For Community (non-Azure team member) PRs:
2525
- [ ] **Security review**: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (`crypto mining, spam, data exfiltration, etc.`)
2626
- [ ] **Manual tests run**: added comment `/azp run azure - mcp` to run *Live Test Pipeline*

‎.vscode/tasks.json‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,20 +14,16 @@
1414
"problemMatcher": "$msCompile"
1515
},
1616
{
17-
"label": "build",
18-
"type": "shell",
17+
"label": "build ToolDescriptionEvaluator",
1918
"command": "dotnet",
19+
"type": "process",
2020
"args": [
2121
"build",
22-
"${workspaceFolder}/AzureMcp.sln",
22+
"${workspaceFolder}/eng/tools/ToolDescriptionEvaluator/ToolDescriptionEvaluator.csproj",
2323
"/property:GenerateFullPaths=true",
2424
"/consoleloggerparameters:NoSummary"
2525
],
26-
"isBackground": false,
27-
"problemMatcher": [
28-
"$msCompile"
29-
],
30-
"group": "build"
26+
"problemMatcher": "$msCompile"
3127
}
3228
]
3329
}

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ The project is organized as follows:
144144
- Include tests in the `/tests` folder
145145
- Ensure all tests pass
146146
- Follow code style requirements
147-
- Run the `eng/tools/ToolDescriptionEvaluator` tool for the new tool description and ensure a result >= 0.4 is achieved
147+
- Run [`ToolDescriptionEvaluator`](https://github.com/Azure/azure-mcp/blob/main/eng/tools/ToolDescriptionEvaluator/Quickstart.md) for the new tool description and obtain a score of `0.4` or more and a top 3 ranking for all related test prompts
148148

149149
## Testing
150150

‎areas/deploy/src/AzureMcp.Deploy/Schemas/deploy-app-topology-schema.json‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
},
3030
"port": {
3131
"type": "string",
32-
"description": "The port number the service uses. Get this from Dockerfile for container apps. If not available, default to \u002780\u0027."
32+
"description": "The port number the service uses. Get this from Dockerfile for container apps. If not available, default to '80'."
3333
},
3434
"azureComputeHost": {
3535
"type": "string",
@@ -44,15 +44,15 @@
4444
},
4545
"dockerSettings": {
4646
"type": "object",
47-
"description": "Docker settings for the service. This is only needed if the service\u0027s azureComputeHost is containerapp.",
47+
"description": "Docker settings for the service. This is only needed if the service's azureComputeHost is containerapp.",
4848
"properties": {
4949
"dockerFilePath": {
5050
"type": "string",
51-
"description": "The absolute path to the Dockerfile for the service. If the service\u0027s azureComputeHost is not containerapp, leave blank."
51+
"description": "The absolute path to the Dockerfile for the service. If the service's azureComputeHost is not containerapp, leave blank."
5252
},
5353
"dockerContext": {
5454
"type": "string",
55-
"description": "The absolute path to the Docker build context for the service. If the service\u0027s azureComputeHost is not containerapp, leave blank."
55+
"description": "The absolute path to the Docker build context for the service. If the service's azureComputeHost is not containerapp, leave blank."
5656
}
5757
},
5858
"required": [

‎docs/e2eTestPrompts.md‎

Lines changed: 234 additions & 234 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)