Integration of Langchain with Llama-Index

The Integration of Langchain with Llama-Index allows us to build smart AI systems that can find and understand information from documents. Llama-Index helps organize and search data efficiently, while Langchain manages how the AI thinks and answers questions step by step. Together, they let developers create apps that can handle complicated questions using both private and public data.

Langchain

Langchain is an open-source framework that helps you build applications using large language models (LLMs). It offers tools to manage prompts, connect different reasoning steps and create smart agents that can use various tools and data sources.

Key Features:

• Run multi-step reasoning tasks.
• Build agents that can use tools like web search or calculators.
• Works with many LLM providers like Gemini, OpenAI and more.

Llama-Index

Llama-Index is a toolkit for building strong search systems on top of unstructured data like PDFs, websites or databases. It helps load, split, embed and search documents, perfect for tasks like retrieval-augmented generation (RAG) or AI-powered data analysis.

Key Features:

• Connects easily to data sources like PDFs, webpages and SQL databases.
• Supports embeddings and vector-based search.
• Offers control over how data is split and retrieved.

Advantages of Integration

• Smarter Information Retrieval: Llama-Index quickly finds the most relevant parts of data and Langchain helps the AI think through it in steps.
• Tool Support: Langchain allows the AI to use multiple tools, like search engines or calculators, alongside document search for better answers.
• Flexible Workflows: Developers can build custom AI agents that combine document knowledge with decision-making logic.
• Works with Any Provider: Compatible with many LLMs and embedding models (OpenAI, Gemini, HuggingFace etc.).
• Built for Scale: Handles large data and complex user questions, making it suitable for real-world apps.

Implementation

Let's create an AI model that loads PDF files using Llama-index, which handles the processing and indexing of the documents. Langchain connects everything, combing document search with the reasoning power of an LLM to answer user questions. To make answers even better, DuckDuckGo is added as a web search tool for pulling in up-to-date information when needed.

What it shows:

• How to handle multiple PDFs at once.
• How AI can understand and compare technical content.
• How document knowledge + real-time web search can improve responses.

Step 1: Install Dependencies

!pip install --upgrade llama-index llama-index-llms-gemini llama-index-readers-pdf llama-index-embeddings-gemini google-generativeai langchain langchain-google-genai duckduckgo-search
!pip install -U duckduckgo-search langchain-community

• llama-index: Document indexing & retrieval
• llama-index-llms-gemini & embeddings-gemini: Gemini-based LLMs and embeddings
• llama-index-readers-pdf: For loading PDF documents
• google-generativeai: Gemini API
• langchain: Orchestrates AI pipelines
• langchain-google-genai: Gemini integration for Langchain
• duckduckgo-search: Adds live web search
• langchain-community: Extra Langchain tools like DuckDuckGo

Step 2: Set Environment and Initialize LLM and Embeddings

from llama_index.core import Settings
from llama_index.embeddings.gemini import GeminiEmbedding
from llama_index.llms.gemini import Gemini
import os
import getpass
os.environ["GOOGLE_API_KEY"] = getpass.getpass(" Enter your Gemini API Key: ")

llm = Gemini(model="models/gemini-1.5-flash")
embed_model = GeminiEmbedding(model_name="models/embedding-001")
Settings.llm = llm
Settings.embed_model = embed_model

• Sets up your Gemini API Key securely.
• llm: Gemini's large language model.
• embed_model: Gemini embedding model.

Step 3: Upload and Load PDF Documents

from llama_index.core import VectorStoreIndex
from llama_index.core.readers import SimpleDirectoryReader
from google.colab import files
uploaded = files.upload()


pdf_files = ["PDF1.pdf", "PDF2.pdf"]
docs = SimpleDirectoryReader(input_files=pdf_files).load_data()
index = VectorStoreIndex.from_documents(docs)

• Let's us upload PDFs.
• Reads and loads them into memory.
• Creates a vector index for semantic search.

Step 4: Prepare the Query Tool

from llama_index.core.tools import QueryEngineTool, ToolMetadata

ai_engine = index.as_query_engine(similarity_top_k=3)

query_tool = QueryEngineTool(
    query_engine=ai_engine,
    metadata=ToolMetadata(
        name="ai_pdfs",
        description="Answers questions based on the uploaded AI PDF documents."
    ),
)

tool = query_tool.to_langchain_tool()

• Converts our vector index into a search tool.
• Sets metadata for easier identification.
• Makes it Langchain-compatible so it can be used by agents.

Step 5: Add Web Search Tool

from langchain_community.tools import DuckDuckGoSearchRun

search_tool = DuckDuckGoSearchRun()
tools = [tool, search_tool]

• Adds DuckDuckGoSearch as a secondary tool.
• Allows the agent to search the web for fresh information.
• Combines both PDF search and web search in a single list of tools.

Step 6: Build Langchain Agent

from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.agents.tool_calling_agent.base import create_tool_calling_agent
from langchain.prompts import ChatPromptTemplate
from langchain.agents import AgentExecutor

system_context = "You are an AI expert. Answer questions using the uploaded AI-related PDFs or the web."
prompt = ChatPromptTemplate.from_messages([
    ("system", system_context),
    ("placeholder", "{chat_history}"),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

llm_agent = ChatGoogleGenerativeAI(model="gemini-1.5-flash")
agent = create_tool_calling_agent(llm_agent, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent, tools=tools, verbose=True, return_intermediate_steps=True)

• Defines the agent's role and instructions.
• Uses a Gemini LLM inside Langchain.
• Connects the agent with both tools: PDF search and web search.
• Wraps everything in an agent executor that can run queries.

Step 7: Interactive Querying and Summarization

question = "What are the main contributions of these AI papers?"
response = agent_executor.invoke({"input": question})
print(response['output'])
summary = ai_engine.query("Can you summarize the first uploaded AI document?")
print(summary)

• Runs a direct query to the document engine.
• Retrieves a summary of document.

Real-World Application

• Research Assistants: AI that can read, summarize and compare reports, research papers or industry standards.
• Financial & Legal Insights: Extract key facts from reports or legal documents and combine them with real-time web news.
• Enterprise Knowledge Search: Chat-based tools to explore internal documents, using smart agents and retrieval.
• Healthcare Q&A: Answer complex medical questions using both stored medical texts and up-to-date guidelines from the web.
• Customer Support Automation: Build AI assistants that can pull answers from product manuals, FAQs, support tickets and even search the web, giving users accurate, context-aware responses instantly.