Forwards and Backwards RAG: Retrieval-Augmented Generation Explained

Retrieval-Augmented Generation (RAG) is a technique that combines large language models (LLMs) with external knowledge sources to improve the accuracy and relevance of generated responses. Recently, the Mayo Clinic introduced the name 'backwards RAG', which enhances the traditional approach with an extra safety step. So let's quickly recap RAG and then explain backwards RAG and when you would use it.

What is RAG?

In standard RAG, when a user asks a question, the system first retrieves relevant documents from a knowledge base. These documents are then provided as context to the LLM, which generates an answer based on both the retrieved information and its own knowledge.

If you've used a commerical chatbot LLM like ChatGPT recently you'll have probably seen it searching the web to help it generate the answer - this is an example of RAG. Any tool that provides extra information to the LLM to help it generate a better answer is a form of RAG.

Forwards RAG: User input flows from top left to bottom right, with additional sources collected in the background.

Here the Knowledge Base is a system for retrieving relevant documents (usually 'hybrid search' that combines keyword search, vector search, and limitations like 'between these two dates'). The actual search terms are determined by the LLM, based on the user query. The LLM uses the information from the Knowledge Base to generate an answer, which is then returned to the user.

What is Backwards RAG?

The results of RAG is an answer that is grounded in many different sources of information. However, this can still lead to hallucinations if the LLM generates incorrect information based on the retrieved documents. 'Forwards RAG' took many different documents, combined them, and then generated an answer.

'Backwards RAG' reverses this process, taking the answer, breaking it down, and then checking the parts against the retrieved documents. We do this to ensure that the answer is grounded in the retrieved knowledge, and to surface any supporting or contradicting evidence. However, simply giving the LLM the generated answer and then asking it 'how true is this?' isn't enough, as we want granularity, clarity, and sources for the evidence.

To achieve granularity we first break the generated answer down into individual claims. Each claim is then checked against the retrieved documents to see if it is supported or contradicted by the evidence. Our overall confidence in the answer is then our lowest confidence in any of the claims (or you may choose to weight this some other way). We prevent low-confidence answers from reaching the user, instead re-generating them or highlighting any inconsistencies in the source documents.

Backwards RAG: The generated answer from Forwards RAG is broken into claims, each of which is verified against the knowledge base and supporting documents.

The validation step is again done by the LLM, which only has to handle one claim at a time and again has access to the Knowledge Base. Depending on your use case you may have it output (per claim):

Confidence score
Supporting evidence snippets
Contradicting evidence snippets

We are, of course, relying on the LLM to be able to accurately assess the claims against the retrieved documents. But this is a powerful safety step that can help prevent hallucinations and ensure that the generated answer is grounded in the retrieved knowledge.

Comparison

RAG: Use retrieved documents to generate answers.
Backwards RAG: Verify generated answers against retrieved documents.

Applications

You can use Backwards RAG as a safety net for any application that requires high confidence in the generated answers.