Django RAG — Knowledge Base

A multi-tenant Django application for ingesting documents and knowledge sources, building vector embeddings, and answering questions via Retrieval-Augmented Generation (RAG).

What it does

• Ingests documents (PDF, DOCX, Markdown, TXT, images via Tesseract OCR) and external sources (Wikipedia topics, web URLs, pasted text).

• Chunks text, embeds it, and stores vectors in PostgreSQL with pgvector.

• Answers questions through an AI model, grounding responses in retrieved chunks and showing sources.

• Retrieves with hybrid search (vector + keyword full-text), fuses the two rankings, and re-ranks with a cross-encoder before answering.

• Falls back to live web search (Wikipedia / DuckDuckGo) when the knowledge base has nothing relevant — opt-in per company.

• Supports real-time streaming chat, voice transcription, and text-to-speech.

• Operates as a multi-tenant SaaS: companies, roles, permissions, and per-company model providers (Ollama, OpenAI, Anthropic, OpenRouter, Groq).

• Exposes a public HTTP API for third-party chat integrations, authenticated with per-company API keys.

Requirements

Services

Environment variables

.env is git-ignored and is not shipped with the repository — create it at the project root yourself. .env.production is also git-ignored, so treat the block below as the authoritative list. Values are read once at startup via core/settings.py; anything already present in the real environment wins (os.environ.setdefault).

SECRET_KEY=
DEBUG=1
ALLOWED_HOSTS=127.0.0.1,localhost,testserver

POSTGRES_DB=django_rag
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_HOST=localhost
POSTGRES_PORT=5432

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_EMBED_MODEL=qwen3-embedding:0.6b
OLLAMA_LLM_MODEL=llama3.2:3b
OLLAMA_MAX_TOKENS=512
OLLAMA_EMBED_NUM_GPU=0

GROQ_API_KEY=
GROQ_LLM_MODEL=openai/gpt-oss-20b
GROQ_BASE_URL=https://api.groq.com/openai/v1

BANGLA_TTS_BACKEND=google-cloud

TESSERACT_CMD=C:\Program Files\Tesseract-OCR\tesseract.exe
ENABLE_LLAMAINDEX_INGEST=1
EMBED_DIM=1024

Q_CLUSTER_TIMEOUT=900
Q_CLUSTER_RETRY=1200

Notes:

• OLLAMA_EMBED_NUM_GPU=0 keeps embeddings on CPU so they never compete with the GPU-resident LLM for VRAM (swapping causes 8–45s cold-load latency per query).

• OLLAMA_LLM_MODEL deliberately defaults to a non-thinking instruct model. Some Ollama builds ignore think: false for Qwen3 and emit chain-of-thought as plain answer text with no <think> tags, which cannot be stripped.

• BANGLA_TTS_BACKEND=google-cloud requires google-cloud-texttospeech plus GOOGLE_APPLICATION_CREDENTIALS. If it is unavailable, Bangla falls back to gTTS automatically; English always uses gTTS.

• EMBED_DIM must match the Chunk.embedding column and the LlamaIndex vector table. Changing it requires a migration and a full re-ingest.

• GROQ_API_KEY is optional. When set it becomes a deployment-level secondary fallback for local Ollama.

Python

Dependencies are pinned in requirements.txt (note: the file is UTF-16 encoded). Notable packages:

• Django 6.0.7

• pgvector + psycopg2-binary

• LlamaIndex (core, Ollama embeddings/LLM, PGVector store, workflows, file readers)

• django-q2 (async task queue)

• faster-whisper (voice transcription)

• pypdf, python-docx, beautifulsoup4, lxml, pytesseract (ingestion)

• FlashRank (re-ranking)

• google-cloud-texttospeech / gTTS (TTS)

• pytest + pytest-asyncio (tests)

Setup

1. Create a PostgreSQL database with the pgvector extension enabled.

1. Install Python dependencies.

1. Create .env at the project root and fill in the variables above.

1. Run migrations:

python manage.py migrate

1. Start the development server:

python manage.py runserver

1. In a second terminal, start the task worker. Ingestion is asynchronous — without this, documents stay pending forever:

python manage.py qcluster

1. Open /accounts/register/ to register a company (there is no page at /accounts/ itself), then / to upload documents.

Ollama

Pull the required local models:

ollama pull llama3.2:3b
ollama pull qwen3-embedding:0.6b

Production deployment guidance lives in OllamaSetup.md (a local, git-ignored document — it is not part of the repository).

Architecture

Apps

• core — Django project configuration, settings, and URL routing.

• accounts — Authentication, multi-tenant companies, RBAC roles/permissions, model providers, and company membership approval flow.

• knowledgebase — Document and topic management, RAG ingestion pipeline, question answering (standard and streaming), voice, and a secondary LlamaIndex index.

• chat_api — Public API-key-authenticated chat endpoints (/api/v1/), the per-company chat configuration (system prompt, web-search toggle), and the API key management screens.

Multi-tenant model

All knowledge-base data is scoped to a Company. Every Document, Topic, and Chunk belongs to a company. Users belong to exactly one company through a Membership that carries a Role (the relation is OneToOne). Permissions are enforced at the view level via the require_permission decorator, and querysets are narrowed by DocumentManager.for_user() / TopicManager.for_user(). Superusers have no Membership, bypass tenant scoping, and act across companies through an "acting company" stored in the session.

Data models

Async processing

Ingestion runs in the background via django-q2 using PostgreSQL as the broker (OrmQ). Document states: pending → processing → done | failed | cancelled. Ingestion acquires a per-document PostgreSQL advisory lock, re-checks the cancelled status between stages, and clears the LlamaIndex rows on delete so the secondary index stays consistent.

Retrieval pipeline

knowledgebase/rag.py — the full path from question to answer:

1. Embed the question with the local Ollama embedding model (1024 dims).

1. Vector search — pgvector cosine distance over Chunk (HNSW index), top 30 candidates.

1. Keyword search — PostgreSQL full-text (tsvector + GIN index), top 30.

1. Relevance guard — if neither signal returns anything, or there are no keyword hits and the best cosine distance exceeds RAG_RELEVANCE_DISTANCE, skip straight to the web fallback.

1. Fuse the two ranked lists with Reciprocal Rank Fusion (k=60).

1. Re-rank the fused pool with a FlashRank cross-encoder (ms-marco-MiniLM-L-12-v2) and keep the top RAG_TOP_K (5).

1. Web fallback (opt-in) — if the best chunk looks unrelated to the question, fetch 1–2 live pages and place them ahead of the weak KB chunks.

1. Answer with a strict context-only prompt that cites source titles, optionally prefixed by the company's custom system prompt. Only the last 10 conversation messages are sent.

Tunable constants at the top of the retrieval section in rag.py:

Text is chunked at 200 words with 40 words of overlap (chunk_text).

Permissions

Permission codes (fixed catalog, seeded by migration — never created by hand):

Default roles created for each company: Company Admin, Manager, Contributor, Viewer. Company Admin is protected from deletion so a company always keeps at least one admin.

API endpoints

Knowledge base UI

Accounts & management

Public API (/api/v1/)

Authenticate with an Authorization: ApiKey <key> header. Keys are created on the API-keys screen and only the SHA-256 hash is stored.

debug: true adds a debug block describing the web-search decision. Note this re-runs the web fetch, so it costs an extra round trip.

AI providers

Configured per company through the UI or admin. Supported provider types:

• Local Ollama

• OpenAI

• Anthropic (Claude)

• OpenRouter

• Groq

Chat model and embedding model are decoupled. The LLM is chosen by the company's active ModelProvider, but embeddings are always produced by the local Ollama embedding model (OLLAMA_EMBED_MODEL) regardless of tenant configuration. This keeps every vector in the knowledge base comparable — mixing embedding models in one column would silently degrade retrieval.

If no provider is active, the deployment falls back to local Ollama, with an optional Groq secondary when GROQ_API_KEY is set. Cloud providers get a local Ollama fallback with a notice prepended to the answer.

api_key values are stored via EncryptedCharField, which signs and base64-encodes the value with SECRET_KEY using django.core.signing. This is obfuscation, not encryption — the plaintext is recoverable from the database, and rotating SECRET_KEY invalidates every stored key.

Web search

Off by default, enabled per company via CompanyChatConfig.allow_web_search.

When the knowledge base returns nothing relevant, the pipeline queries Wikipedia's opensearch API and then DuckDuckGo's HTML endpoint, fetches up to 2 pages, strips navigation/scripts, and injects the text as leading context (capped at max_chars=12000 per page). The fetched pages are surfaced in the source list as their URLs.

Vector search

Embeddings are produced by the local Ollama model, coerced to the column dimension (EMBED_DIM, default 1024) via Matryoshka dimensions where the model supports it. Retrieval uses cosine distance on pgvector with an HNSW index (vector_cosine_ops, m=16, ef_construction=64). A secondary LlamaIndex PGVector index (llamaindex_vectors) powers only /ask-llamaindex/ and can be disabled with ENABLE_LLAMAINDEX_INGEST=0.

Testing

# Django test runner — provider, source-filtering, web-fallback merge tests
python manage.py test

# Public API tests (pytest style)
pytest chat_api/tests.py

chat_api/tests.py uses pytest fixtures and requires pytest-django, which is not currently in requirements.txt and there is no pytest.ini/conftest.py. Until that is added, those tests cannot be collected.

Development notes

• Migrations live under knowledgebase/migrations/ and accounts/migrations/.

• The EncryptedCharField avoids django-cryptography due to compatibility with Django 6.0.

• Tesseract path is configurable via TESSERACT_CMD.

• Session chat history is capped at 24 messages (12 exchanges); only the last 10 messages are sent to the model. The public API accepts up to 12 exchanges.

• Document.file_type is inferred from the extension. Only png/jpg/jpeg are treated as images — other image formats fall through to plain-text reading.

• STATIC_ROOT is not set, so collectstatic (used by the deploy workflow) will fail until it is configured.

• Document.topic_group stores the topic name as a string rather than a foreign key, so renaming a topic does not update already-grouped documents.

• The deploy workflow (.github/workflows/deploy.yml) runs on push to main: git reset --hard, pip install, migrate, collectstatic, then restarts the knowledgebase and knowledgebase-qcluster systemd services.