From the course: Hands-On AI: Introduction to Retrieval-Augmented Generation (RAG)

Different embedding models - Python Tutorial

From the course: Hands-On AI: Introduction to Retrieval-Augmented Generation (RAG)

Start my 1-month free trial

Different embedding models

“

- [Instructor] We've talked about the importance of embedding models. Let's learn how we can use different embedding models for different use cases. There are two main differences between embedding models. The first is the type of data that it embeds. In an earlier video, I said that you can use any deep learning model that is trained on the type of data that you want to embed as an embedding model, and this is where that idea comes in. You can use embedding models that work on images, text, video, audio, or even multiple types of data. The second difference is in the length of embedding. Embeddings are almost always in the magnitude of hundreds to thousands, but many embedding models differ in the exact length of their vector embeddings. Different embedding models produce embeddings of different length due to their architecture. Remember, embeddings are just the output from the second to last layer of a deep learning model. Due to the way that vector math works, only embeddings of the same length can be compared. As a general rule, longer embeddings can help you achieve more fine green semantics, but are more expensive, while shorter embeddings are less specific, but also less costly. As a final note on embedding models, remember that they are highly dependent on their training data. It is not just the length of the embedding, it's also the data that went into the training. As such, you may need certain domain specific embedding models for certain tasks. For a deeper dive into embeddings, you should take my advanced rag applications with vector databases course.

Contents