From the course: Claude with Amazon Bedrock by Anthropic

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Model-based grading

Model-based grading

From the course: Claude with Amazon Bedrock by Anthropic

Start my 1-month free trial Buy for my team

Model-based grading

“

The next thing we're going to implement inside of our prompt evaluation workflow is a grading system. Now as a reminder, a grader is going to take in some output coming from our model, and then our hope is that the grader is going to give us some kind of objective signal. It might be a number or a true or false value. It could be really anything, but very commonly, very frequently, you're going to see a number output between 1 and 10, where 10 means that we got a very high quality output and 1 means we got a very low quality output. Again, that's not not a requirement, we don't have to get numbers out of these graders, but it's a very common practice you're going to see very often. There's three different kinds of graders that we're going to discuss in this video. Code, Model, and Human. Let's first figure out what code-based graders are all about. With a code-based grader, we're going to take the output from our model and feed it into a snippet of code that you and I author. Inside…

Contents