A local, privacy-first Retrieval-Augmented Generation (RAG) chatbot for querying personal knowledge bases in Vietnamese. Built on LangChain, ChromaDB, and a locally-running Ollama LLM — no external API calls required after setup.

• Vietnamese-optimized retrieval using intfloat/multilingual-e5-base with correct query/passage prefixes
• Dual chunking strategy — Markdown documents split by header hierarchy, PDFs by recursive character splitting
• Max Marginal Relevance (MMR) retrieval for diverse, non-redundant context chunks
• Multi-turn conversation memory via ConversationalRetrievalChain with a sliding window of the last 5 turns
• Source citations shown inline after every answer — file name and section heading

1. Project Overview
2. Key Features
3. Tech Stack & Architecture
4. Methodology & Pipeline
5. Project Structure
6. Examples
7. Installation & Setup
8. Usage
9. Configuration
10. References

Managing personal notes and lecture PDFs in Vietnamese makes standard English-centric RAG solutions a poor fit — most default embedding models perform poorly on Vietnamese text, and typical chunking strategies ignore Markdown structure entirely.

This project builds a fully local RAG chatbot that:

• Accepts Vietnamese PDF and Markdown documents as a knowledge base
• Retrieves semantically relevant chunks using a multilingual embedding model tuned for Vietnamese
• Generates grounded answers via a locally-running qwen3:4b model through Ollama
• Maintains conversational context across follow-up questions
• Exposes a clean Streamlit chat UI with document upload and source attribution

• PDF support via PyPDFLoader — preserves page-level metadata for citation
• Markdown & TXT support via TextLoader
• Bulk ingestion through the ingest.py CLI script
• On-the-fly ingestion via the Streamlit sidebar file uploader (no restart needed)

• Markdown documents: split first by heading hierarchy (#, ##, ###) using MarkdownHeaderTextSplitter; sections exceeding 512 characters fall back to RecursiveCharacterTextSplitter
• PDFs & plain text: RecursiveCharacterTextSplitter with chunk size 512, overlap 50
• Header metadata is preserved on every chunk for accurate citations

• Embedding model: intfloat/multilingual-e5-base with mandatory "query: " / "passage: " prefixes (required by the E5 model family for correct similarity scores)
• Vector store: ChromaDB with SQLite persistence — no separate server needed
• Search strategy: Max Marginal Relevance (MMR), fetching 10 candidates and returning the 5 most diverse

• ConversationalRetrievalChain reformulates follow-up questions into standalone queries before hitting ChromaDB
• ConversationBufferWindowMemory keeps the last 5 conversation turns in context
• reasoning=False on the Ollama LLM disables Qwen3's thinking mode for faster responses

• Streamlit chat interface with st.chat_message and st.chat_input
• Per-session chain and memory (multiple browser tabs get independent conversations)
• Vectorstore cached across sessions via @st.cache_resource
• Collapsible Sources expander under each answer showing file name and heading

User Query
    │
    ▼
ConversationalRetrievalChain
    │ reformulates question using chat history
    ▼
MultilingualE5Embeddings  ──►  ChromaDB (MMR, k=5)
                                    │ top-5 chunks
    ┌───────────────────────────────┘
    ▼
ChatOllama (qwen3:4b)
    │ answer + source_documents
    ▼
Streamlit UI  ──►  citations expander

data/ (PDFs, MDs, TXTs)
    │
    ├── PyPDFLoader       → page-level Documents
    └── TextLoader        → full-file Documents
    │
    ▼
Chunker
    ├── .md  → MarkdownHeaderTextSplitter → RecursiveCharacterTextSplitter (fallback)
    └── rest → RecursiveCharacterTextSplitter (512 tokens, 50 overlap)
    │
    ▼
MultilingualE5Embeddings ("passage: " prefix)
    │
    ▼
ChromaDB (persisted to chroma_db/)

User question
    │
    ▼
Question condenser  (uses chat history to rewrite follow-ups)
    │
    ▼
E5 query embedding ("query: " prefix)
    │
    ▼
ChromaDB MMR search  (fetch_k=10 → return k=5)
    │
    ▼
Prompt: [system] + [retrieved chunks] + [question]
    │
    ▼
qwen3:4b (Ollama, local)
    │
    ▼
Answer + source document metadata → Streamlit

• E5 prefix handling: the MultilingualE5Embeddings wrapper subclasses HuggingFaceEmbeddings and prepends "passage: " during indexing and "query: " at query time — required for correct cosine similarity scores with this model family.
• Per-session chain: the LangChain chain (and its memory) lives in st.session_state, so each browser tab has independent conversation history while sharing the same cached vectorstore.
• MMR over top-k: reduces retrieving multiple nearly-identical chunks from the same document section, improving answer diversity.

RAG/
├── app.py                      # Streamlit entry point
├── ingest.py                   # CLI bulk ingestion script
├── config.py                   # All constants (paths, models, chunk sizes)
├── requirements.txt
├── .env.example                # Copy to .env to override defaults
├── data/                       # Drop your PDFs and Markdown files here
├── chroma_db/                  # Auto-created — ChromaDB persistence
└── src/
    ├── ingestion/
    │   ├── loader.py           # PyPDFLoader + TextLoader wrappers
    │   └── chunker.py          # Dual chunking strategy (MD + recursive)
    ├── retrieval/
    │   ├── embeddings.py       # MultilingualE5Embeddings with E5 prefixes
    │   └── vectorstore.py      # ChromaDB init + MMR retriever factory
    └── chain/
        └── rag_chain.py        # ConversationalRetrievalChain + memory

• Python 3.11+ (tested on 3.14.3)
• Ollama installed and running
• ~2 GB disk space for the embedding model download

# 1. Clone / navigate to the project
cd "AI Agents Projects\RAG"

# 2. Create and activate virtual environment
python -m venv venv
.\venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure environment (optional — defaults work for local Ollama)
cp .env.example .env

# 5. Pull the LLM via Ollama
ollama pull qwen3:4b

Drop PDF, Markdown, or TXT files into the data/ folder, then run:

python ingest.py
# or specify a custom directory:
python ingest.py --data-dir path/to/your/docs

streamlit run app.py

Open http://localhost:8501 in your browser.

All tuneable parameters are in config.py:

• LangChain Documentation
• ChromaDB Documentation
• Ollama
• intfloat/multilingual-e5-base on HuggingFace
• Qwen3 Model Family
• Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.