Skip to content
Architecture Lab

Architecture Lab

Retrieval-Augmented Generation pipeline

Grounding LLM answers in a private document library: ingestion, hybrid retrieval, re-ranking and cited answers.

  • RAG
  • LLMs
  • pgvector
  • Hybrid search

Illustrative reference design built from general engineering practice. It is not the architecture of any employer's or client's production system.

Why it matters

An LLM on its own answers from training data and can't see a user's documents. RAG retrieves the relevant passages first and asks the model to answer only from them, with citations. This is the direction of the AI tutor in PrepForge Academic Hub, whose ingestion pipeline (parse → chunk → embed) feeds the retrieval layer.

Diagram

RAG pipeline: ingestion and query pathsDocuments are parsed, chunked, embedded and indexed. A question is embedded with the user's access scope, hybrid retrieval finds candidate chunks in the index, a re-ranker keeps the best, a prompt is built, the LLM answers, and a citation check runs before the answer is returned.Ingestion (background)Query (per request)DocumentsPDF, DOCX, slides, textParse & chunkStructure-awareEmbed chunksBackground workerVector + text indexpgvector, full-textUser questionEmbed queryScoped to the userHybrid retrievalVector + keywordRe-rankKeep the best fewAnswer + citationsCitation checkRefuse if unsupportedLLM providerVia a gatewayBuild promptPassages as datacandidates
  • Request / call
  • Asynchronous
Illustrative
Diagram as text

Components

  • Documents (input or output): PDF, DOCX, slides, text
  • Parse & chunk (service): Structure-aware
  • Embed chunks (queue or worker): Background worker
  • Vector + text index (data store): pgvector, full-text
  • User question (input or output)
  • Embed query (service): Scoped to the user
  • Hybrid retrieval (service): Vector + keyword
  • Re-rank (service): Keep the best few
  • Answer + citations (input or output)
  • Citation check (control point): Refuse if unsupported
  • LLM provider (external system): Via a gateway
  • Build prompt (service): Passages as data

Connections

  • Documents → Parse & chunk
  • Parse & chunk → Embed chunks (asynchronous)
  • Embed chunks → Vector + text index
  • User question → Embed query
  • Embed query → Hybrid retrieval
  • Vector + text index → Hybrid retrieval (candidates)
  • Hybrid retrieval → Re-rank
  • Re-rank → Build prompt
  • Build prompt → LLM provider
  • LLM provider → Citation check
  • Citation check → Answer + citations

How it flows

  1. Ingest

    Uploaded files are validated, parsed and split along headings, slides and tables so each chunk is a coherent passage.

  2. Index

    A background worker embeds each chunk and stores the vector beside its text, owner and document version.

  3. Scope the query

    The question is embedded and every lookup is filtered to documents the user is allowed to read.

  4. Retrieve

    Vector similarity and keyword search run together, so exact terms such as error codes still match.

  5. Re-rank

    A re-ranker orders the candidates by relevance and only the top few go into the prompt.

  6. Generate

    The prompt carries the passages as quoted data with their sources; the model must answer from them.

  7. Check

    Each claim must cite a retrieved passage. If nothing relevant was found, the answer says so instead of guessing.

Boundaries

User data boundary

Retrieval is owner-scoped in the database query itself, not filtered afterwards, so one user's chunks never reach another user's prompt.

Model provider boundary

Only the selected passages leave the system, through one gateway that handles keys, quotas and provider choice.

Ingestion vs query

Heavy parsing and embedding run in background jobs; the request path only reads the index.

Failure handling

No relevant passages
Return "not found in your documents" rather than letting the model answer from general knowledge.
LLM provider error or timeout
Retry once, then fall back to another provider through the gateway; show a clear error if none responds.
Document replaced mid-processing
Jobs act only on the current document version and discard results for an outdated one.
Instructions hidden in a document
Retrieved text is marked as untrusted data in the prompt, and the system prompt tells the model never to follow it.

Trade-offs

pgvector inside PostgreSQL

Gain
One system of record for documents, chunks and embeddings, and access rules in plain SQL.
Cost
A dedicated vector database may scale further for very large corpora.

Hybrid instead of vector-only search

Gain
Exact identifiers and rare terms are found reliably.
Cost
Two result lists must be merged and tuned.

A re-ranking step

Gain
Fewer, better passages, so prompts are smaller and answers more focused.
Cost
Extra latency on every question.

Next design

AI-assisted code review pipeline