About RAG

RAG

i keep forgetting what RAG means, so i’m writing it down here.

RAG stands for Retrieval-Augmented Generation. It lets an AI system search for relevant information before answering. First, a retriever finds useful passages. Then, the language model uses them as context to generate a response.

The main idea is retrieval: finding relevant information to use in an answer. It isn’t about the AI compressing information.

Usage

RAG is often used in chatbots that need to answer questions using sources like technical documentation. At least this is where I found the most use of it.

The library analogy

• Knowledge base: the library
• Documents or notes: the books
• Search index: the library catalogue
• Retriever: the librarian
• LLM: the person answering the question
• RAG: the librarian finds relevant pages and gives them to the person answering the question

Why use RAG at all?

A language model’s knowledge can be outdated or may not include private information. RAG gives it extra context by retrieving relevant material from a knowledge base, such as recently updated documentation. The model can then use that information alongside what it already knows to answer the question.