Improving RAG with Contextual Compressors and Filters

KEEP IN TOUCH | THE GEN AI SERIES

PROBLEM

1. In RAG systems, the retrieved context contains a large amount of irrelevant information. Often, only a small portion contributes to the overall answer.
2. There are instances where questions require synthesis of facts from multiple retrieved chunks.

Precision is crucial. And introducing extraneous information into the context learning window is an issue.

So, two major problems.

SOLUTION

Contextual Compression addresses the challenge of unpredictable queries in document storage systems.

When data is ingested into the system, specific queries are unknown, leading to relevant information being buried in documents containing extensive irrelevant text. Passing entire documents through applications can result in costly calls to Large Language Models (LLMs) and poorer responses.

The process of Contextual Compression involves:

1. Base Retriever: Retrieves diverse pieces of information.
2. Document Compressor: Filters and processes the retrieved information, extracting only what is essential to answer the question.