A local-first RAG pipeline that refuses to hallucinate.
Built on LangGraph, Ollama, and pgvector.
Each agent has exactly one job. The graph is compiled once and reused — no rebuilds per request.
Generates a HyDE-style hypothetical answer, creates multiple rewritten queries for multi-query retrieval, classifies intent, and decides whether retrieval is even needed.
Handles greetings, app-help questions, and conversational prompts that should never pretend to cite documents. A dedicated escape valve from the retrieval pipeline.
Runs hybrid dense + sparse retrieval across all rewritten queries and the HyDE answer. Deduplicates while preserving all retrieval metadata, then reranks with a cross-encoder.
The anti-hallucination gate. Asks whether retrieved context is sufficient, produces a faithfulness score, and routes weak evidence straight to refusal — no sneaking through.
Generates cited answers with inline [Source N] references. Post-checks citation validity. If grounding checks fail, the user gets an explicit refusal — never a silent empty answer.
True hybrid retrieval. Every returned chunk is annotated with diagnostics. Every diagnostic is surfaced in the UI.
pgvector similarity search with nomic-embed-text (768 dims). Annotates each doc with dense_rank and dense_score.
Corpus-level BM25 across the full indexed collection — not just dense candidates. Cache invalidated on every ingest. Annotates sparse_rank and sparse_score.
Reciprocal Rank Fusion merges both ranked lists: RRF(d) = Σ 1/(k + rank_i(d)). Annotates rrf_score and retrieval_channels.
cross-encoder/ms-marco-MiniLM-L-6-v2 reranks the fused top-K. Final reranker_score determines what reaches generation.
Every feature is designed around one principle: never guess.
View Source Code →Ingest PDFs and web URLs. PDF ingestion captures title, author, page count, and page numbers. Web ingestion is section-aware — it tracks h1/h2/h3 headings and groups content under heading paths.
Chunk sizes are measured in tokens via tiktoken — not characters. Each chunk is prefixed with structural hints. Chunk IDs are collision-resistant: they encode source name, page marker, doc index, chunk index, and a content hash.
Every returned document carries a full diagnostic trail: dense_rank, dense_score, sparse_rank, sparse_score, rrf_score, reranker_score, and retrieval_channels. These are surfaced directly in the UI — no black box retrieval.
Two independent grounding layers. First: the Validation Agent checks faithfulness score against threshold (0.60). Second: the Generation Agent post-validates that every [Source N] reference maps to an actual retrieved chunk. Both can independently trigger refusal.
All models run locally via Ollama. No API keys. No data leaves your machine. Generation model (qwen2.5:7b) is configured for temperature 0.1 — deliberately low for factual RAG. Validation model (phi4-mini:3.8b) is optimized for reasoning at small scale.
You need Docker, Ollama, and Python 3.11+. That's it.
git clone https://github.com/kedar49/RAG-Application.git
cd RAG-Application
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env# Start pgvector (PostgreSQL + pgvector)
docker compose up -d
# Pull required Ollama models
ollama pull qwen2.5:7b # ~4.7 GB
ollama pull phi4-mini:3.8b # ~2.5 GB
ollama pull nomic-embed-text # ~274 MBstreamlit run app.py
# Open http://localhost:8501