From the course: Python for AI Projects: From Data Exploration to Impact

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Setting up LLMs

Setting up LLMs - Python Tutorial

From the course: Python for AI Projects: From Data Exploration to Impact

Start my 1-month free trial Buy for my team

Setting up LLMs

“

- [Instructor] We've explored our data sources and discussed how to build context, and now it's time to take a closer look at the architecture of our AI system and the key components we'll use to bring it to life. To power search and discovery over product information and user reviews, we'll use a technique called semantic search. Instead of looking for exact keyword matches, semantic search retrieves documents based on meaning. For example, surfacing reviews that mention scenic views, even if the user searched for beautiful landscapes. To make this work, we'll use a sentence embedding model, specifically a BERT-based sentence transformer from the Hugging Face Transformers library. This model converts each document into a vector that captures its meaning in a high-dimensional space, allowing us to compare documents and queries based on their semantic similarity. We'll then store these vectors in a Faiss index, which is short…

Contents