From the course: AI Toolkit Essentials for Visual Studio Code

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Adding a custom evaluator: Finish and test

Adding a custom evaluator: Finish and test - Visual Studio Code Tutorial

From the course: AI Toolkit Essentials for Visual Studio Code

Start my 1-month free trial Buy for my team

Adding a custom evaluator: Finish and test

“

- [Instructor] With the initial setup of the custom evaluator in place, we're ready to edit the Python code. Before doing this, we want to make sure that the column names we specified are correct in the variable section. Once we go to the next step, we cannot go back and change 'em in AI Toolkit. The only things we'll be able to change later are the name, description, if we are using the local runtime and the Python code itself. With that checked, we're going to start by pressing the Generate Evaluator Code button. Now this has opened up a new Python code window in Visual Studio, and we can see that we have two parameters for our two columns, and a third one added for **kwargs, and this is basically general purpose arguments. But as far as I can tell, the **kwargs parameter dictionary is never used, but we should leave it in place as there's warnings that tells us not to change the parameters. So we're not going to change the parameters, but we have our ground_truth and response being…

Contents