From the course: LLM Foundations: Vector Databases for Caching and Retrieval Augmented Generation (RAG)

Partitions in Milvus

From the course: LLM Foundations: Vector Databases for Caching and Retrieval Augmented Generation (RAG)

Partitions in Milvus

“

Partitions are an important concept in Milvus that helps in performance. What is a Milvus partition? A collection in Milvus can be split up into multiple partitions. What is special about partitions? All data in a partition are stored physically together. Each collection gets created with the default partition called _default. When data is inserted into a collection, it goes into the default partition. However, we can specify a partition name during data insert to send the data to a specific partition. We can either create partitions manually and assign them manually during inserts, or we can use partition keys to automatically allocate data during inserts to specific partitions. Partitions help optimize storage and retrieval. This allows data to be grouped for storage using popular filter fields like tenant, customer, or device IDs as partition keys. When retrieving data for a given filter, fewer data blocks are retrieved from disk as they are stored together, resulting in better efficiency.

Contents