Building a Retrieval-Augmented Generation (RAG) System with Oracle Database 23AI, OCI and LLM(s): Powering Up Information Retrieval with AI

Greetings, database enthusiasts! Today, we're diving into the exciting world of Retrieval-Augmented Generation (RAG) systems.  Here I 'am blogging again. This time with the motivation of Oracle Database 23 AI's vector capabilities, that we leveraged while building ourselves a Sales Assistant. While leveraging the power of Oracle 23AI, we also used OCI 's and Cohere's large language models (and integrations), and Flask for building our solution. This AI bases Sales Assistant is in beta version, but we can make it production ready by putting a little more effort, and making it reach perfection.

Basically, our sales assistant takes questions from the user and provides answers by using the LLM models in the backend. However; the answers generated by the LLM modes are always kept in the context, and this is done by the help of RAG (and some orchestration done via our python code.)

The value here is that, we used Oracle Database 23AI for our vector store. So we didn't position a separate database for storing vector embeddings and doing vector similarity searches.

You can extrapolate that and think of what you can do with these types of new features of Oracle Database 23AI.

Anyways.. What about RAG?

Imagine a system that can not only find relevant information but also craft insightful answers based on that knowledge. That's the essence of RAG. It takes information retrieval a step further by using generative models to create comprehensive responses to user queries.

Our Project: 

In this project, we created a custom RAG system that leverages three key players:

Oracle 23AI: Oracle's new version database acts as our knowledge repository, storing documents and their corresponding vector embeddings (think of them as condensed representations capturing the document's meaning).

Cohere: We'll tap into Cohere's arsenal of LLMs, like command-r-plus, for answer generation. These models are masters at weaving words into coherent and informative responses.

Flask: This lightweight web framework serves as the user interface, allowing users to interact with our system and receive answers to their questions.

The Deep Dive: How It Works

Query Embeddings: When a user asks a question, the system transforms it into an embedding using Cohere. This embedding becomes the key to unlocking relevant information.

Knowledge Retrieval: The system dives into the Oracle 23AI database, wielding the power of vector similarity search. It compares the query embedding with stored document embeddings to identify the most relevant documents – think of it as finding the closest matches in the knowledge vault.

Refining the Results: Not all retrieved documents are created equal. We utilize Cohere's reranking model to sort these documents by their true relevance to the user's query, ensuring the most pertinent ones are at the forefront.

Answer Generation: Now comes the magic! Cohere's LLM takes center stage. It analyzes the query and the top-ranked documents, crafting a comprehensive answer that incorporates both the user's intent and the relevant retrieved information. 

Serving Up the Answer: Finally, the user receives the answer, along with the most relevant documents to provide context and transparency.

Why Oracle 23AI?

Here's why Oracle 23AI is the perfect partner for our RAG system:

Vector Powerhouse: Its vector datatype enables efficient storage, indexing, and retrieval of document embeddings, crucial for speedy searches.

Scalability: As our system grows, Oracle 23AI can handle the increasing volume of data with ease.

A Word on Overcoming Challenges

During our project, we encountered a minor problem: the Frankfurt region on Oracle Cloud Infrastructure (OCI) didn't support the specific Cohere model we needed. --Note that, for some part of the work, we reached LLM models through OCI (via its services -- integration), and for some other part of it (like the text generation), we reached LLM models directly from our code..-- So, we switched to the Chicago region, which provided seamless integration. Just a reminder, sometimes a quick regional shift can save the day!

The Future of RAG: A World of Possibilities

RAG systems hold immense potential to revolutionize information retrieval. By combining retrieval-based approaches with generative models, we can create systems that understand user intent, provide comprehensive answers, and constantly learn and improve.

Ready to Build Your Own RAG System?

This blog post serves as a springboard for your RAG exploration. With the power of Oracle 23AI, Cohere, and Flask, you can create a system that empowers users to unlock the true potential of information. Stay tuned for future posts where we delve deeper into the code and implementation details!

As always, feel free to leave any questions or comments below.