Dive into the world of advanced language understanding with Advanced_RAG. These Python notebooks offer a guided tour of Retrieval-Augmented Generation (RAG) using the Langchain framework, perfect for enhancing Large Language Models (LLMs) with rich, contextual knowledge.

In the Basic RAG architecture, the query follows a clear path from user input to the final generated response. The process generally involves the following steps:

1. User Query: The user provides a natural language query.
2. Query Encoding: The query is transformed into a vector representation using an encoder.
3. Retrieval: The encoded query is used to search a knowledge base (e.g., dense vectors in a vector store) to retrieve relevant documents.
4. Response Generation: The retrieved documents are then used by a language model to generate a response, which is presented to the user.

The advanced RAG architecture introduces more intricate components that enhance the overall system:

1. Query Construction: Incorporates multiple query variations to improve retrieval accuracy.
2. Multi-step Retrieval: Involves using different retrieval strategies, such as Multi Query Retriever, to obtain diverse document sets.
3. Document Ranking: Implements ranking mechanisms to select the most relevant documents.
4. Adaptive Generation: The final response is generated by dynamically adapting to the context provided by the retrieved documents.

The Multi Query Retriever structure optimizes the retrieval process by using multiple queries to enhance the document search. This method provides diverse responses from various sources and selects the best ones for the generation process.

In Self-Reflection-RAG, the model reflects on its own responses during the generation process to improve accuracy. This self-reflective mechanism helps the model refine its outputs and minimize errors in its final response.

Agentic RAG introduces an autonomous agent that navigates the retrieval and generation process. This agent evaluates the retrieved documents and dynamically chooses the best information to construct a comprehensive response.

Adaptive Agentic RAG builds on the Agentic RAG architecture by incorporating adaptability. The agent adjusts its behavior based on context and user interaction, optimizing the retrieval and response generation dynamically.

In Corrective Agentic RAG, the agent not only adapts but also corrects its retrievals and responses in real-time. It identifies potential errors or ambiguities and rectifies them to ensure an accurate final output.

The LLAMA 3 Agentic RAG Local setup integrates the capabilities of the LLAMA 3 model into the RAG framework. It allows for local processing, making it a more secure and efficient method for information retrieval and generation. This model employs an advanced agentic approach, utilizing the robust understanding of the LLAMA 3 model to navigate through large datasets and generate precise responses.

Below is a detailed overview of each notebook present in this repository:

• 01_Introduction_To_RAG.ipynb
  • Basic process of building RAG app(s)
• 02_Query_Transformations.ipynb
  • Techniques for Modifying Questions for Retrieval
• 03_Routing_To_Datasources.ipynb
  • Create Routing Mechanism for LLM to select the correct data Source
• 04_Indexing_To_VectorDBs.ipynb
  • Various Indexing Methods in the Vector DB
• 05_Retrieval_Mechanisms.ipynb
  • Reranking, RaG Fusion, and other Techniques
• 06_Self_Reflection_Rag.ipynb
  • RAG that has self-reflection / self-grading on retrieved documents and generations.
• 07_Agentic_Rag.ipynb
  • RAG that has agentic Flow on retrieved documents and generations.
• 08_Adaptive_Agentic_Rag.ipynb
  • RAG that has adaptive agentic Flow.
• 09_Corrective_Agentic_Rag.ipynb
  • RAG that has corrective agentic Flow on retrieved documents and generations.
• 10_LLAMA_3_Rag_Agent_Local.ipynb
  • LLAMA 3 8B Agent Rag that works Locally.

Enhance your LLMs with the powerful combination of RAG and Langchain for more informed and accurate natural language generation.