From the course: LLM Foundations: Vector Databases for Caching and Retrieval Augmented Generation (RAG)

Build an index

From the course: LLM Foundations: Vector Databases for Caching and Retrieval Augmented Generation (RAG)

Build an index

“

We added data to the collection in the previous video, but before we can use it for semantic search, we need to create an index on the vector field. To create an index, we first need to set some parameters for the index. The metric type is used to indicate the distance measure that should be used to compare the vectors. This can be L2 or IP or cosine. Do note that the same metric type should be used when doing queries on this vector field. The index_type is set to IVF_FLAT. We discussed the options here in the previous chapter, and list indicates the number of clusters or buckets to create in this index. Higher values of this parameter can lead to better efficiency but lower search effectiveness. To create the index, we use the same collection object. We specify the field on which to create the index and the index parameters. We can check on the index creation process by using the index_building_process method as shown here. Let's run this code now. We can see that the index has been successfully created. Going to the Collections page for this course_db database, we can see the courses_list collection. On clicking on the collection name, we can see the schema and the indexes. In order to do a data preview or a data query, we first need to load the collection into memory. To do this, we go back to the collection list. Click on the unloaded status of the collection. Then we hit the "Load" button to load the collection. Now it is loaded. We can then click on the collection name and then go to Data Preview to take a preview into the contents of this collection.

Contents