Updated scoring metrics and renamed utility for evaluating tool descriptions (#965)

* Updated scoring metrics and results

* Renamed the utility to ToolDescriptionEvaluator

* Added licensing header to .cs files

Author: vcolin7

.github/PULL_REQUEST_TEMPLATE.md

@@ -20,7 +20,7 @@
     - [ ] Updated `README.md` documentation
     - [ ] Updated command list in `/docs/azmcp-commands.md`
     - [ ] Updated test prompts in `/e2eTests/e2eTestPrompts.md`
-    - [ ] For new or modified tool descriptions, ran the `eng/tools/ToolDescriptionConfidenceScore` tool and obtained a result >= 0.4
+    - [ ] For new or modified tool descriptions, ran the `eng/tools/ToolDescriptionEvaluator` tool and obtained a result >= 0.4
 - [ ] 👉 For Community (non-Azure team member) PRs:
     - [ ] **Security review**: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (`crypto mining, spam, data exfiltration, etc.`)
     - [ ] **Manual tests run**: added comment `/azp run azure - mcp` to run *Live Test Pipeline*

CONTRIBUTING.md

@@ -144,7 +144,7 @@ The project is organized as follows:
    - Include tests in the `/tests` folder
    - Ensure all tests pass
    - Follow code style requirements
-   - Run the `eng/tools/ToolDescriptionConfidenceScore` tool for the new tool description and ensure a result >= 0.4 is achieved
+   - Run the `eng/tools/ToolDescriptionEvaluator` tool for the new tool description and ensure a result >= 0.4 is achieved
 
 ## Testing
 

docs/new-command.md

@@ -1381,12 +1381,11 @@ catch (Exception ex)
    - **Maintain alphabetical sorting in e2eTestPrompts.md**: Insert new test prompts in correct alphabetical position by Tool Name within each service section
 
 5. Tool Description Quality Validation:
-    - Test your command descriptions for quality using the validation tool before submitting:
+    - Test your command descriptions for quality using the validation tool located at `eng/tools/ToolDescriptionEvaluator` before submitting:
 
       - **Single prompt validation** (test one description against one prompt):
 
         ```bash
-        cd eng/tools/ToolDescriptionConfidenceScore
         dotnet run -- --validate --tool-description "Your command description here" --prompt "typical user request"
         ```
 
@@ -1411,7 +1410,7 @@ catch (Exception ex)
         dotnet run -- --prompts-file my-prompts.json
 
         # Tools:
-        # Use JSON format (same as eng/tools/ToolDescriptionConfidenceScore/tools.json):
+        # Use JSON format (same as eng/tools/ToolDescriptionEvaluator/tools.json):
         dotnet run -- --tools-file my-tools.json
 
         # Combine both:
@@ -1448,7 +1447,7 @@ catch (Exception ex)
         ```
 
     - Custom tools file format:
-      - Use the JSON format returned by calling the server command `azmcp-tools-list` or found in `eng/tools/ToolDescriptionConfidenceScore/tools.json`.
+      - Use the JSON format returned by calling the server command `azmcp-tools-list` or found in `eng/tools/ToolDescriptionEvaluator/tools.json`.
 
 6. Live Test Infrastructure:
    - Use minimal resource configurations for cost efficiency

eng/scripts/Test-ToolSelection.ps1

@@ -60,7 +60,7 @@ $RepoRoot = $RepoRoot.Path.Replace('\', '/')
 
 Push-Location $RepoRoot
 try {
-    $toolSelectionPath = "$RepoRoot/eng/tools/ToolDescriptionConfidenceScore"
+    $toolSelectionPath = "$RepoRoot/eng/tools/ToolDescriptionEvaluator"
 
     if (-not (Test-Path $toolSelectionPath)) {
         Write-Host "⏭️  Tool selection test not found at $toolSelectionPath - skipping"