Step by step: Advanced chatbot workflow in n8n

- [Instructor] So now that we have the data ingestion and data retrieval workflows set up, let's build our actual chatbot that will make use of this data retrieval flow. So create a new workflow and add a chat trigger here. For now, we can just close this field and add our AI agent node. Make sure that this is connected to the chat trigger, and then let's add a system prompt to it. Again, the system prompt will tell the agent basically how to behave, but also that it should call a tool in order to retrieve knowledge. So let's go back to our GitHub repository. And in this repository under "advanced chatbots," you will find the system prompt. Now, let's take a look at this briefly. The system prompt is actually really similar to the system prompt that we had before. There are just three key differences. The first difference is that this system prompt tells the agent to get information that is found in the GET HR policy tool. So this means it's not looking for information inside the prompt, inside the FAQ, but it needs to call a tool, which is called GET HR policy, in order to retrieve knowledge that it'll then use to answer the question. The second modification is that we added this line over here: "If the user asks a very broad question, "try to narrow it down first "so you can provide more compact answers." Why are we doing this? Well, because one of the problems of our RAG architecture, and RAG architectures in general, is that those open-ended questions can't really be answered. For example, if a user asks for all company benefits, then this can't really be answered unless there's some text in the source documents that lists all those benefits in one place. But otherwise, because we always limit the amount of chunks that we retrieve, we can't really be certain. So that's why whenever the user asks one of those very broad questions, we want the chatbot to ask back and actually narrow that down so we can then retrieve information that is actually correct and relevant to that question. And then finally, we also added citations. So citations basically means that when the chatbot generates the answers based from the retrieved chunks, it should also give the citation or show the citation where this information came from as a footnote here. And the way we do this in this case is just by providing this as instructions here in the prompt and then giving an example of how the citation should look like, in this case, the footnote followed by the document name and then the URL to that document. So that's the system prompt we're going to use. So let's copy everything from here and go back to n8n, and then paste that into the system message. All right, let's close this AI agent node for now, and let's add a chat model. Important note here: At this stage, you can really use any model. I will be using Gemini again, but just because we have the API key for that. There isn't really now any connection between the model that is holding the conversation with the user and the model that we use for the embedding because, remember, the whole retrieval logic is now outsourced to a separate workflow. What we just do here is to send a keyword to this workflow, and then this workflow will retrieve all the relevant information from our vector store. So you can really pick any model that you like from here. I will go with Gemini 2.5 Pro because it's one of the best models here. Make sure that you select options, sampling temperature, and set that back to zero. I don't know why n8n has a high sampling temperature here out of the box. So let's close this for now and add a memory to our agent. Again, let's just add a simple memory here. Just the context window length to a message to something like 20. Again, this is just the number of messages or past interactions that the bot can remember as part of an existing conversation. Again, feel free to adjust that to your preference. And finally, tool use. Now, this is the most important piece because as a tool, we want to define the workflow that we just set up. So let's choose 'call an n8n workflow tool' from here. And first, let's give this tool a little description. In this case, we are going to be explicit about the name. We will call it 'GET HR policy' and then add a little description, like "This tool will search the HR knowledge base." So this description essentially tells our chatbot that this is the tool to get information from the HR knowledge base. To keep things tidy, let's also assign this here as a name for this node. Now let's pick the workflow that we are using for this, search for retrieval, to pick the chatbot data retrieval workflow that we just defined. You'll see that the workflow inputs are listed here automatically. We need to click this little AI icon to tell n8n that this query parameter should be filled in automatically by our chat model. So that's the setup we need over here. And let's close this one for now. And basically, that's it. That's the minimal setup we need to build an AI agent that is now able to pull documents from our internal knowledge base. So let's try it out. Click 'open chat,' and let's greet this agent with a little 'hi.' So you can see the response over here: "Hello, I'm your HR assistant. "How can I help you with HR policies today?" And the interesting thing is now that you can see that this agent actually did not use any tool. The agent just used the Gemini chat model in order to generate the response. But let's see what happens if we ask it about something that it needs to search in our HR policy documents. So for example, we could ask it something like, "Where do I apply for vacation?" And you can now see that it used the knowledge tool here in order to get that information. It's also shown in the traces of the AI agent. So let's take a look at what happened here. So if you click on this GET HR policies tool, you'll see that the search query, in this case, was "vacation application." And this search query was defined by our chat model. And if you click on output, you can see the chunks of information that are being retrieved for this particular search query. Now, Gemini, in this case, our chat model, used this information to put it into the context of the current conversation and generate the final answer from that, which in this case is "You can apply for vacation by following these steps." And you can see that in the end, we got a citation of where this information came from. Now, this is great because it shows us that our chatbot is working as expected. Now, let's ask it about something that is actually not stored in our knowledge database. Like for example, "How can I get a MacBook?" You can see that the agent is now going back to the GET HR policy tool again and sending the query "how to get a MacBook." But the thing is, there isn't really a term 'MacBook' stored anywhere in our documents. So what happens is that our search just returns the closest chunks to that. And very close chunks to that are things that, for example, contain "personal laptops" because it's semantically really close to a MacBook. So that means this is the information that our chatbot now has in order to generate the answer from. And in this case, it answered something like this: "Based on the IT policies, "here's the information regarding the company equipment. "The company provides the necessary computer. "For special equipment, you should talk to IT, "and personal laptops are generally not allowed for work." Now, you might get a slightly different response, but the key insight here is that our chatbot was able to answer that question even though the specific keyword 'MacBook' was not in our original knowledge source. So this is why these agents are so powerful, because they allow to have these natural-sounding interactions without actually being restricted to doing a plain keyword search. We still search for keywords, but the results that we get are even richer than that. That's what we call a semantic search. Now that we confirmed our bot is working, let's give this a name. Let's, for example, call it "advanced chatbot" and try this chatbot out in the real world.