From the course: Hands-On AI: Build a RAG Model from Scratch with Open Source

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Setting up a database and retrieving vectors and files

Setting up a database and retrieving vectors and files

From the course: Hands-On AI: Build a RAG Model from Scratch with Open Source

Start my 1-month free trial Buy for my team

Setting up a database and retrieving vectors and files

“

- [Instructor] Once we've established how we can generate vector embeddings and have an understanding of how much information they should ideally encapsulate, we're ready to modularize the process to create a database of text with their accompanying vector embeddings. We'll use these text-vector pairs in real-time when running our RAG models. It's crucial to keep our end goal in mind. We're building a database which we want to query to access documents of interest by searching across the vector embeddings of those documents. We have to store the data in a way where each document or idea, which will be saved in the vector embeddings, is not much less or much more than a single idea. Since we're dealing with textual data, one natural way to break up our database entries is into sentences, because as discussed earlier, each vector embedding holds one idea, and an idea is approximately what we store in one sentence. This…

Contents