Skip to content

Conversation

@vritant24
Copy link
Member

@vritant24 vritant24 commented Jul 22, 2025

cc @lramos15

Allow specifying a file to configure Open AI compatible models not available in existing flows and api.
the file is of the form:

{
  "<model id>": {
   "name": "<model name>",
   "version": "<model version>",
   "capabilities": {
    "supports"?: {
     "parallel_tool_calls"?: <boolean>,
     "streaming"?: <boolean>,
     "tool_calls"?: <boolean>,
     "vision"?: <boolean>,
     "prediction"?: <boolean>
    },
    "limits"?: {
     "max_prompt_tokens"?: <number>,
     "max_output_tokens"?: <number>
    }
   },
   "url": "<endpoint URL>",
   "apiKeyEnvName": "<environment variable name for API key>"
  },
  ...
}

and can be specified using the --model-config-file option.

example

An example run would be:
npm run simulate --model=custom-model-id --model-config-file=./custom-config.json

the custom-config.json:

{
	"custom-model-id": {
		"name": "custom model name",
		"version": "2025-04-07",
		"capabilities": {
			"supports": {
				"parallel_tool_calls": true,
				"streaming": true,
				"tool_calls": true,
				"vision": true,
				"prediction": false
			},
			"limits": {
				"max_prompt_tokens": 128000,
				"max_output_tokens": 4096
			}
		},
		"url": "https://custom-model.com/api",
		"apiKeyEnvName": "API_KEY"
	}
}
@vritant24 vritant24 marked this pull request as ready for review July 22, 2025 20:11
@vritant24 vritant24 requested a review from lramos15 July 23, 2025 00:20
lramos15
lramos15 previously approved these changes Jul 23, 2025
@vs-code-engineering vs-code-engineering bot added this to the July 2025 milestone Jul 23, 2025
amunger
amunger previously approved these changes Jul 23, 2025
@vritant24 vritant24 dismissed stale reviews from lramos15 and amunger via b93b7e1 July 23, 2025 16:03
@lramos15 lramos15 enabled auto-merge July 23, 2025 16:05
@lramos15 lramos15 added this pull request to the merge queue Jul 23, 2025
Merged via the queue into microsoft:main with commit a4dbf63 Jul 23, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

4 participants