Brief Documentation: AI Knowledge Assistant (RAG)

What is it?

RAG (Retrieval Augmented Generation) is an architectural pattern that answers questions using a language model, but only on top of documents the organization owns. The model does not answer from memory. It answers from passages retrieved at question time, and the answer carries the sources it came from.

The Problem it Solves

A language model on its own has two defects that make it unusable for internal knowledge. It does not know company documents, and it cannot show where an answer came from. Fine tuning a model on those documents is slow, expensive, and stale the day a document changes.

Retrieval solves both. The documents stay in a store you control, the index updates when a file changes, and every answer can be traced back to the passage that produced it.

How the Architecture Works

The pattern has two paths that run at different times.

1. The Ingestion Path (write time)

   • The Ingestion API accepts a file, saves the original in the Document Store, and puts one message on the Ingestion Topic.
   • The Indexing Worker picks up that message, splits the document into passages, asks the external model provider for embeddings, and writes them to the Knowledge Index.
   • A document that fails indexing lands in the dead letter queue instead of disappearing.
   • Ingestion is asynchronous on purpose. Uploading 500 documents must not block the person who uploaded them.

2. The Answer Path (read time)

   • The Chat Service authenticates the user against the Identity Provider and finds out what that user is allowed to see.
   • It retrieves the relevant passages from the Knowledge Index, sends them to the model provider together with the question, and returns the answer with its sources.
   • The Answer Cache short circuits questions that were already asked. The Conversation Store keeps the history, so a follow up question has context.