The rise of large language models (LLMs) like ChatGPT has greatly enhanced efficiency across a wide range of everyday business tasks through automation. But for highly-constrained processes, such as responding to Request-for-Proposals (RFPs), challenges remain. LLMs can produce hallucinations, and tracing the source of generated responses is often difficult, making auditability a persistent concern.
For RFPs, automated systems must generate precise, verifiable answers that a human can quickly review and correct. This requires identifying the relevant sections of the source document for as many questions as possible and correctly mapping them to the appropriate context – capabilities that LLMs alone cannot reliably provide.
To address these challenges, we introduced an improved Retrieval Augmented Generation (RAG) architecture designed specifically for automated RFP completion. This system is designed to enhance the relevance and accuracy of generated content, significantly minimizing manual effort in reviewing and correcting answers.
Key components of an improved RAG system
Our RAG system is built to help businesses quickly find the right information and generate accurate responses. The proposed improvements are two fold:
- Smart Search (Hybrid Embedding): We combine two types of “understanding” of documents: one that grasps context and meaning (dense contextual embeddings), and another that looks at statistical patterns in the text (sparse statistical embeddings). Together, they make it much less likely for the system to pull irrelevant info and much better at finding the right answers.
- Document Page Finder: Once we’ve pulled relevant information, this tool tells you exactly which page in the source document it came from, making it easier to check the answer and verify accuracy.
Practical deployment of this solution demonstrates its suitability not only for automatic RFP completion, but also as a general architecture for domains where efficiency, accuracy, and verifiable responses are paramount.
The system, step by step
Every accurate answer starts with the right building blocks. Here’s a step-by-step look at the four core components that power the system.
- Hybrid Retriever: Think of this as a smart search engine that looks through your documents and picks the top five chunks of text most likely to answer your question.
- Vector Database: This is where all the document “summaries” (embeddings) and important metadata are stored so the system can find information efficiently.
- Generator: Using a powerful AI model (we used Mistral 8x7B, but any equivalent model should work well), it takes the retrieved chunks and turns them into clear, accurate responses — perfect for RFPs.
- Document Page Finder: Finally, it maps every chunk of text back to its original page in the document, so you always know where your answers came from.
Validated RFP performance
When we tested our system with real-world RFPs, it performed well. The hybrid search approach made answers more accurate and almost completely eliminated cases where the system gave responses that didn’t match the question. The Document Page Finder could also precisely point to the exact page in the source document for every piece of retrieved text.
In practice, the enhanced system has proven to be fast, reliable, and easy to scale, making it a great fit for businesses that need precise answers and clear traceability. Looking ahead, we plan to continue optimizing its retrieval speed and response generation.
Want to learn more about RAG? Join FreeClimb Research on Friday, December 12 at 11:00 a.m. CST for a live webinar where our experts will showcase a real-world RAG use case and share actionable tips for improving content retrieval accuracy.