Improving RAG Systems

KEEP IN TOUCH | THE GEN AI SERIES

RAG makes LLMs useful by giving them factual context to use while answering queries. RAG is easy to prototype, but very hard to productionize. It is probably the single most effective way to use LLMs as of today (3rd Jan, 2024)

Now improving means increasing the proportion of queries for which the system:

1. Finds the proper context &
2. Generates and appropriate response.

Let’s try to understand how we can improve RAG pipelines. Note: Given how fast the space is moving, it is impossible to provide and exhaustive, or perfectly up to date, list of best practices.

1. DATA CLEANING

If the data is confusing, in substance or layout, then the RAG system will suffer. If we are using data with conflicting or redundant information, our retrieval will struggle to find the right context. And when it does, the generation step performed by the LLM may be suboptimal.

We can use LLM to create summaries of all the documents provided as context. The retrieval step can then first run a search over these summaries, and dive into the details only when necessary.

2. HYBRID SEARCH