RAG Chatbot in AmazonBedrock

日本語 | English

RAG (Retrieval Augmented Generation) chatbot is an AI-powered chatbot that enhances AI language models by incorporating documentation or/and source URLs as the data source. This technique enables the chatbot to provide accurate and up-to-date responses, even if the required information is the latest or unavailable online. While non-RAG chatbots rely on pre-trained data and may struggle to offer current or specialized knowledge, the RAG chatbots can retrieve information from a specific knowledge base or external sources. This makes RAG chatbots particularly useful for applications, where up-to-date information is critical, such as legal or government updates, product releases, or personalized content.

Table of Contents

• Used Tools and Concepts
• Objectives
• Workfkow
• Document
• Demo
• Data-Driven Test Results
• License

Used Tools and Concepts:

The AWS services I used in this project were Amazon Bedrock, S3, and OpenSearch Serverless. Key concepts include storing data in S3, adding source URLs through Web Crawler, creating a Knowledge Base, requesting access to AI models, how chatbot generates responses using AI models and Knowledge Base, and utilizing vector stores for efficient retrieval.

Objectives

I. Use Case 1 – Personalized Q&A Chatbot

Since my best friend recently got engaged, I wanted to give them something unique, meaningful, and a little humorous gift. Then, I decided to build an application integrated with an RAG chatbot that answers questions about him, allowing his fiancée to learn more about him in a fun way. This not only makes them laugh but also helps them start their new life together with a deeper understanding.

II. Use Case 2 – Automated Immigration Updates

Whenever the Government of Canada announces new rules or updates regarding temporary residents and Permanent Residency (PR), I usually spend 15 to 20 minutes searching through multiple documents and sources to verify the information’s accuracy. I realized that the RAG chatbot could streamline this process by providing reliable updates directly from trusted sources if I could successfully integrate with public web links in the Knowledge Base. This would significantly reduce my search time while ensuring access to accurate and up-to-date information.

Workflow

1. Add Data Source
   • Use Case 1
     • Store Data in S3 - Stored relevant documentation in Amazon S3 as the chatbot's data source.
   • Use Case 2
     • Add Source URLs - Instead of S3, added source URLs as the data source through Web Crawler.
2. Create a Knowledge Base - Created a Knowledge Base in Amazon Bedrock to enable retrieval-based AI responses.
3. Request Access to AI Models - Selected and requested access to specific AI models, enabling the AI models to convert search results into human-like text.
4. Configure Vector Store - Used OpenSearch Serverless for vector-based search, allowing efficient retrieval of relevant information.
5. Sync Knowledge Base - Synchronized the data from S3 to the Knowledge Base to ensure the following three keys:
   • Ingesting: Bedrock takes the data from S3.
   • Processing: Bedrock chunks and embeds the data.
   • Storing: Bedrock stores the processed data in the vector store (OpenSearch Serverless).

By following these flows, the AI chatbot is transformed into the RAG chatbot, significantly improving its ability to generate informed responses.

Document

Please reference a document to see more details.

Demo

I. Use Case 1 – Personalized Q&A Chatbot

II. Use Case 2 – Automated Immigration Updates

Data-Driven Test Results

I. Use Case 1 – Personalized Q&A Chatbot

I compared response speed by testing identical prompts while storing 15 vs. 5 documents in S3. As a result, I discovered that the response speed might hypothetically depend on the volume of stored documents, according to the following data:

• 1st Prompt: 4.61 sec. (15 docs) → 3.26 sec. (5 docs)
• 2nd Prompt: 5.70 sec. (15 docs) → 5.50 sec. (5 docs)
• 3rd Prompt: 6.18 sec. (15 docs) → 5.78 sec. (5 docs)
• 4th Prompt: 6.81 sec. (15 docs) → 6.80 sec. (5 docs)

On average, the response speed was 0.49 seconds faster when storing fewer documents. This is probably because the AI chatbot took longer to process and retrieve information from the larger volume of documents.

However, while reducing the number of documents could improve the performance of response speed, it may also negatively affect the quality of responses. Therefore, finding the right balance between speed and knowledge depth is crucial for optimizing chatbot performance.

II. Use Case 2 – Automated Immigration Updates

I used to spend approximately 15 to 20 minutes searching for reliable sources and reading through numerous documents every time the IRCC announced new updates regarding immigration, especially Express Entry rules for PR. However, the RAG chatbot can now retrieve updates directly from added source URLs and provide the relevant information with reliable sources, reducing my search time by 66.67 % to 75% – from 15-20 minutes to only 5 minutes (including the time to verify accuracy).

License

This project is licensed under the MIT License. See the LICENSE file for details.