From the course: Complete Guide to Evaluating Large Language Models (LLMs)

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Measuring retrieval-augmented generation (RAG) systems

Measuring retrieval-augmented generation (RAG) systems

From the course: Complete Guide to Evaluating Large Language Models (LLMs)

Start my 1-month free trial Buy for my team

Measuring retrieval-augmented generation (RAG) systems

“

- In our last section, we talked about measuring agentic systems, most notably on their tool selection criteria. Let's turn our attention now towards another very common task of AI, retrieval augmented generation, or RAG. Now RAG is something we've actually talked about already, has two main components, the retriever, the R, and the generator, the G. The retriever is trying to pull information from a database, i.e., information retrieval. The generator is basically how conversational is the bot. So like agentic AI has several components to test, like its tool selection and its final outputs to the users, RAG is similar. We can test independently their ability to pull relevant information and their ability to use that information to answer a question. So starting with the retriever, as information retrieval generally is, it's an embedding/classification task, because what you're saying is, of all these hundreds of thousands of documents, the embedding similarity, the cosine similarity,…

Contents