Go back
Entity RAG + LLMs: Powering Enterprise Grade AI
Join us to explore the latest cutting-edge AI technology behind the most accurate and reliable GenAI applications transforming customer service.
Jason Wirth
7/3/2024
What happens when you combine 'Entity-RAG' with 'ChatGPT' - Enterprise grade GenAI at your fingertips.
Let’s explore the key components involved in building highly accurate GenAI applications, that delivers accurate, safe and trustworthy AI made for enterprise use.
Components:
- Retrieval augmented generation (RAG)
- Entity RAG = entity resolution + fuzzy search
- LLMs like ChatGPT
Firstly let’s quickly introduce Retrieval augmented generation (RAG)
RAG is the process of adding more context to a prompt sent to an LLM through the accurate retrieval of company data.
Here is a visual showcasing a simple example with ChatGPT
On the left, we posed a specific business question about an insurance contract to the standard ChatGPT. While ChatGPT provided a response, there were several issues:
- The response was generic and not particularly useful.
- It lacked a contextual understanding of the business use case.
- It failed to fully grasp the complexities involved.
- Despite not having the necessary information, it still provided an answer.
On the right: In addition to the original query, we enriched it by incorporating elements of the business contract that address the original question. This enhancement...
- Delivers a more accurate response tailored to the user's needs.
- Provides ChatGPT with more contextual understanding of the business use case.
- Combines GPT's general knowledge with the specific business domain data.
- Recognizes the user's expectation for a business-specific answer.
This process is automated by RAG to ensure high-standard accuracy and data security. Dive deeper into the RAG technology by downloading our comprehensive RAG guide here.
Now let’s quickly discuss LLMs
Large Language Models (LLMs) such as ChatGPT are revolutionizing the field of natural language processing by emulating human dialogue, impacting both business and public sectors. These models can store and quickly retrieve a vast amount of information, providing reasonably accurate responses to user queries. In the business world, LLMs provide numerous advantages, such as automating repetitive, manual processes to help teams complete business-critical tasks more quickly and accurately.
Together, RAG and LLMs are driving the next wave of business evolution: GenAI
BUT, like all new technology GenAI comes with implementation challenges, and one of the biggest challenges organizations face when implementing GenAI is: a minimal error tolerance.
Errors or hallucinations in GenAI applications occur when the AI provides a wrong or misleading response and if hallucinations persist it’s unlikely AI will be adopted wide scale. A common source of errors in LLMs stems from the quality of the input data. Contaminated, duplicated, or complex data can hinder the LLMs' ability to extract information with the necessary accuracy.
This challenge can be addressed by incorporating Entity-Enhanced RAG into the existing RAG pipeline. Dive into more detail about the why RAG can sometimes fail.
Entity-RAG explained: Entity Resolution & Fuzzy Search
Real-world data is inconsistent, it can be spoiled with spelling mistakes and formatting errors, making it difficult for LLMs to connect the right dots.
Consider this example featuring a fictional character named after Vincent van Gogh, showcasing various possible variations in name spelling, address typos, and differently written email addresses.
Typically, even minor variations in spelling or formatting can challenge LLMs in extracting accurate information. However, this issue can be resolved with the assistance of Entity-RAG.
There are several techniques how:
1. Fuzzy search - Fuzzy matching techniques measure the similarity between two or more strings or data points, which enables the identification of non-exact matches.
2. Geomatching - a concept where spatial or geographical data is used to enhance the retrieval process.
3. Temporal Matching- the technique of incorporating time or temporal data into the retrieval process to enhance the relevance and accuracy.
Working with Tilores.io
Before we dive into examples of entity RAG in action, to integrate Entity-RAG we work with Tilores, who developed Identity resolution software for fraud prevention, KYC & compliance use cases.
Tilores employs entity resolution to generate secure, dynamic graph views of your customer data in real-time, regardless of data volume or the number of data sources.
A workflow for entity infrastructure with Kern and Tilores:
Learn more about Tilores at www.tilores.io
3 different development approaches to showcase the accuracy of entity powered RAG.
All the following strategies were developed on the Kern platform, integrating GPT-4.
We are asking the same question for each question:
"Hi, my name is Mich Buchannon! My goldfish sadly got sick while swimming on the beach yesterday. I could gladly rescue him but now he is not feeling well and can't take our flight back home tomorrow. Is that covered by out insurance?"
Development Strategy 1: Plain RAG.
This approach represents a very simple approach of just data + vector database + LLM.
So looks a little like this:
Application output:
"Based on the information provided in the documents, the rescue of your sick goldfish while swimming on the beach may be covered by the insurance policy under the Search, Recovery, or Rescue Costs section . You should review the specific coverage limits and conditions outlined in your Travel Health Protection Gold or Platinum policy summaries to confirm eligibility for reimbursement."
Answer is really vague. LLM unable to give an excellent answer.
Development Strategy 2: Enhanced RAG
Adding some more magic to our RAG pipeline with LLM transformations plus our custom reranking model.
Reranker model
Application output:
"Based on the information provided in the documents, the insurance policies cover unforeseen health incidents such as sudden illnesses, injuries, or newly developed medical conditions that necessitate immediate attention, including emergency room visits, prescribed treatments, and medical procedures. However, the policies do not cover care provided after the coverage ends. In this case, the sickness of your goldfish while swimming on the beach and its subsequent care may not be covered by the insurance policies mentioned in the documents. It's advisable to review the specific terms and conditions of your insurance policy to determine coverage for this situation."
The answer is much more nuanced and to the point. It is more helpful, although some ambiguity remains.
Development Strategy 3: Enhanced RAG + Tilores
We can now extract names from our customer message.
Application output
"Based on the information provided, Mitch Buchannon currently holds two active insurance plans: the "Local Explorer" plan, valid from January 1, 2023, to January 1, 2025, and the "Global Guard Health Premium" plan, valid from July 15, 2023, to July 15, 2024. Unfortunately, neither the "Local Explorer" nor the "Global Guard Health Premium" insurance policies explicitly cover incidents related to pets or animals, such as a goldfish getting sick. These policies primarily focus on medical emergencies, trip cancellations, accidents, and other travel-related issues for humans. Therefore, the situation with your goldfish not feeling well and affecting your flight back home is not covered under the terms of these insurance plans."
As you can see, each development strategy, including the integration of entity RAG software like Tilores, helps to enhance quality and relevance.
Watch our Webinar
Sign up for our newsletter to get the latest updates on LLM.
Go to newsletter