What is agentic retrieval-augmented generation (RAG)? - Microsoft Copilot Studio Tutorial

(soft swelling music) - What is Agentic RAG? How is it different to basic RAG and how can we use it in our applications? We are going to answer these questions in this fifth lesson of the AI Agents for Beginners course. In this course, we're going to take you from concept to code, covering the fundamentals of building AI agents. And in this short video follows along with the written lesson, including translations and code samples that you can find along the link above and below this video. So let's get started looking at what Agentic RAG is, and we will start looking at basic RAG first. So RAG or retrieval-augmented generation enables LLMs to take a query made from a user and retrieve relevant information from a database. That information is then added to the context given to the LLM to provide more relevant responses outside of just the trading data of the LLM. And while that framework isn't going away, Agentic RAG adds a new level of capability to it. To start on the query side, the agent performs an analysis of the query to create a plan of smaller tasks to use the data sources and tools it will need to answer that query. And because while there is no such thing as a bad question, there are complex questions that require multiple steps and systems to answer. And after that information is retrieved from those systems, the agent can verify if the information is enough to answer that query. If not, it can repeat this process of tool calling until it does. And what is really great about this is that the agent can maintain long-term memory of this process. So it can recall the previous attempts it's tried to answer the query and improve the next time without taking those steps. And this is why I would've called it Adaptive RAG, but no one asked me when we were naming these things. So Agentic RAG it is. That's enough talking. Let's head over to the code and see how to build this. Now we're here at our code editor to look at the Agentic RAG sample. I am using the Semantic Kernel as your search example because we can now create an index using Azure AI search, which we will show you in the setup lesson how to actually set up to get the right credentials. And again, this code is located at the GitHub repo at the top and the bottom of this video, so you can work with yourself on your machine. So the first thing I want to point to is this class prompt plugin that we've created. Basically this allows us to, after the prompt or the request from the user is said, we can actually augment the prompt so that it gives a lot more direction to the agent in terms of being able to retrieve information. So we basically are allowing this augmented prop to come in and it's basically essentially like RAG where we will take the retrieved context, so whatever we retrieved from our Azure search database or index, and then we're going to take the user query and we're going to direct the agent to first review the retrieved context and see if it answers that query. If it doesn't, we're also going to direct it to say, "Try any of the available plugin functions." 'Cause that's again, the power of Agentic RAG. It's not only to do the retrieval part, but to combine those elements to other tools and things to get information to answer the query of the user. So after that, we will make a plugin just so that the agent has another option. In this case, we have this class WeatherInfoPlugin, and this basically just returns a few different locations and an average temperature. So in an ideal world, when the user is asking about temperature, we don't actually have that information in the Azure search, 'cause again, maybe that changes quite often, but we also have a plugin or another tool that retrieves that information. And when a user is making that query, we should be able to combine those things. So now we will also add these documents into our Azure search index. So we have these travel documents. We're going to use our credentials that we set up in the setup chapter. And then we basically have just some three, or sorry, five documents here that has just some basic information around insurance, the destinations that are available and even the benefits of those destinations. Now we're going to actually look at the interaction from the user and the agent. And I've kind of set this up so that we can see how different parts of the application are handling different things in terms of the Agentic RAG. So in this example, the user's asking around, explaining Contoso's travel insurance coverage. And the context that it retrieves is a few different documents. So we've given it directions to pull in the top three. In this case you can see that obviously the first one is really relevant here in terms of really answering the question around travel insurance, but the other ones might be able to support it. So you can always see the context that the agent's operating by using this dropdown. And then you can see the response that the agent gives out. In this case, this is honestly asking about what the average temperature of the Maldives is. So it does a few different things. First, it retrieves that this is a destination available or offered by Contoso. But as you can see here, since the first review or the retrieved context doesn't answer that question, we can actually go back or look at these function calls and it's actually going to make this function call, which is the get_destinations. So again, this is the destination, it's calling with Maldives, and it's going to give back the average temperature of the Maldives. I'm not sure exactly if this is correct, average temperature of Maldives. But again, this is just to show you the ability to first look at the context and see what available tools that it needs to be called. And then the last context or last example is when we need to handle both retrieving from RAG perspective and the function calling. Because like I said, one of the things is that, with Agentic RAG is that we can handle more complex type of queries than this, the basic RAG. So in this case, they're asking for a good cold destination offered by Contoso and what is the average temperature. So there's two things, steps the agent needs to take, right? First to see what destinations Contoso offers and then also the average temperature of that. So if you look at this example, first it's that the RAG context, it retrieves a document that says the popular destinations. So Maldives, Swiss Alps, and African safaris. And then it also makes that function call about the Swiss Alps to see that this is a cold destination and it gives out the nice response, which is, "A good cold destination offered by Contoso is the Swiss alps," with the average. So we've done both a RAG context, as well as a function call. So that was that. Again, you can run this code on your side, play with the questions, maybe add some more documents, really experiment and seeing how the powers of Agentic RAG. But I'll see you in the next lesson.