EVALUATING THE PERFORMANCE OF OFF-THE-BOX RETRIEVAL AUGMENTED GENERATION SOLUTIONS TO PROVIDE RECOMMENDATIONS ON CHOOSING THE BEST SOLUTION

Abstract

This article presents the process of developing an application to provide a comprehensive assessment of existing Retrieval-Augmented Generation (RAG) solutions with a focus on their effectiveness in the financial sector, particularly for M&A companies. RAG systems, which combine search mechanisms with generation capabilities, have shown promise in providing accurate and contextually relevant answers using large amounts of industry-specific data. We created software code that automatically scores the answers and creates a league table to compare these solutions across thirteen key performance indicators. A dataset of questions and answers from actual industry experts was collected, and the ability of each solution to process structured and unstructured financial data was assessed. We also compared the scores from experts and large language models, and concluded on the effectiveness of the study. The findings highlight the strengths and weaknesses of existing RAG systems, provide insight into their applicability and potential to improve decision-making in the financial sector. This study aims to assist organisations in choosing the most appropriate RAG solution for their needs, as well as provide information on future developments in this rapidly evolving field.

The study demonstrates the significant potential of advanced search systems in the field of mergers and acquisitions. The results obtained allow organizations to choose the most suitable RAG solution for their needs, increasing the accuracy and relevance of answers to complex queries, which in turn improves decision-making processes in the financial sector.

This study makes a significant contribution to the development of knowledge about RAG technologies, especially in the context of their application in the financial sector. The results are an important step in understanding and implementing these technologies, which will help provide more accurate, relevant and useful answers to complex financial queries.