Chenwei X.

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,671

190 Comments

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Let's learn more! In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point—MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable—without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. I created it while taking inspiration from Femke Plantinga's post on this topic. If you want to understand this better with code, I have linked my article in the comments. ____ If you want to learn AI/ML engineering, I have put together a free PDF (530+ pages) with 150+ core DS/ML lessons. Get here: https://lnkd.in/gi6xKmDc ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

1,382

48 Comments

New paper "Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluation" with Google DeepMind colleagues Don Metzler and Zhen Qin is now available on arXiv: https://lnkd.in/d3A7AXAJ 🎓 The paper synthesizes existing research and presents novel experiment designs that explore how LLM-based rankers and assistants influence LLM-based judges. 📊 We provide the first empirical evidence of LLM judges exhibiting significant bias towards LLM-based rankers. 🤔 Contrary to some previous findings, our preliminary study does not find evidence of bias against AI-generated content.  🗺️ We offer initial guidelines and a research agenda to ensure the reliable use of LLMs in IR evaluation.

116

If you’re an AI engineer or an AI leader looking to level up your skills in LangChain and LangGraph, this one is for you. I’m teaming up with Arvind Narayanamurthy for a beginner–intermediate, 1.5 hour hands-on workshop where you’ll learn how to move from simple prompts to real AI applications and agentic workflows. Why this matters - Most engineers know how to prompt. Few know how to orchestrate. - LangChain and LangGraph are becoming the backbone of production AI systems. - If you want to build or lead AI engineering teams, you need these mental models. What you’ll learn - Foundations of LLM orchestration and where LangChain fits - Essential building blocks: chat models, prompting patterns, structured outputs, LCEL - Tool calling with MCP adapters - RAG basics end to end - LangGraph workflows and agentic architectures - Two live labs: a micro-app and a simple agent - Quick dive into LangSmith for debugging The workshop is recorded, and everyone gets the replay. If you want practical, real-world skills to build modern AI systems, this is the one to sign up for. Grab your seats: https://lnkd.in/eW2WZdcf

328

36 Comments

If you’re an AI engineer or an AI leader looking to level up your skills in LangChain and LangGraph, this one is for you. I’m teaming up with Arvind Narayanamurthy for a beginner–intermediate, 1.5 hour hands-on workshop where you’ll learn how to move from simple prompts to real AI applications and agentic workflows. Why this matters - Most engineers know how to prompt. Few know how to orchestrate. - LangChain and LangGraph are becoming the backbone of production AI systems. - If you want to build or lead AI engineering teams, you need these mental models. What you’ll learn - Foundations of LLM orchestration and where LangChain fits - Essential building blocks: chat models, prompting patterns, structured outputs, LCEL - Tool calling with MCP adapters - RAG basics end to end - LangGraph workflows and agentic architectures - Two live labs: a micro-app and a simple agent - Quick dive into LangSmith for debugging The workshop is recorded, and everyone gets the replay. If you want practical, real-world skills to build modern AI systems, this is the one to sign up for. Grab your seats: https://lnkd.in/eW2WZdcf Image credits: Nano Banana

262

23 Comments

Just getting started with LLMs? Don’t let the jargon slow you down. This 5-part LLM Glossary is your shortcut to mastering the fundamentals👇 1️⃣ Model Types • Foundation Model: Pretrained on large-scale, diverse datasets to learn general capabilities. • Instruction-Tuned Model: Further trained to follow user instructions precisely. • Multi-modal Model: Handles multiple input/output formats like text, images, and audio. • Reasoning Model: Optimized for logic, problem-solving, and step-by-step thinking. • Small Language Model (SLM): Lightweight models for fast, efficient, on-device tasks. 2️⃣ Training & Fine-Tuning • Pretraining: Initial learning phase using massive, diverse data sources. • RLHF (Reinforcement Learning with Human Feedback): Aligns model behavior with human preferences. • DPO (Direct Preference Optimization): Uses ranked preferences instead of rewards for training. • Synthetic Data: AI-generated data used to supplement real datasets. • Fine-Tuning: Targeted retraining on specific domains or tasks. • LoRA / QLoRA: Techniques for efficient low-resource fine-tuning. • Guardrails: Rules or filters applied to enforce safety, ethics, and compliance. 3️⃣ Prompt Engineering • System/User Prompts: Define model behavior and user context in a session. • Chain-of-Thought (CoT): Prompts that guide the model to reason step-by-step. • Few-Shot / Zero-Shot Learning: Demonstrating tasks with few or no examples. • Prompt Tuning: Optimizing prompts via training for specific outcomes. • Context Window: The total number of tokens the model can "remember" per interaction. 4️⃣ Inference & Generation • Temperature: Adjusts output randomness; lower values are more deterministic. • Max Tokens: Limits the number of tokens in a generated response. • Seed: Controls reproducibility of outputs. • Latency: Time taken by the model to respond. • Hallucination: When the model generates factually incorrect but plausible-sounding information. 5️⃣ Retrieval-Augmented Generation (RAG) • Retrieval: Fetching relevant data from external sources before generation. • Semantic Search: Search based on meaning, not just keywords. • Embeddings: Vector representations of text used for similarity matching. • Chunks: Dividing documents into smaller segments for better retrieval. • Vector Databases (VectorDBs): Stores embeddings for fast and accurate search. • Reranking: Rescoring retrieved documents to prioritize relevance. • Indexing: Structuring data for efficient retrieval during generation. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

232

36 Comments

Among other things, my team has been focusing on AI research for agents. We just blogged about a recent project: the Test-Time Diffusion Deep Researcher (TTD-DR). This framework uses a Deep Research agent to draft and revise its own drafts using high-quality retrieved information. This approach achieves new state-of-the-art results in writing long-form research reports and completing complex reasoning tasks. The work achieves better results than any other deep research agent we compared to. It also shows that self-evaluation and self-improvement are must-have components of a successful agent. https://lnkd.in/gBZpVMQy

297

8 Comments

Speeding up PyTorch performance usually requires the use of intermediate optimization frameworks, or writing low-level optimized kernels. This can be burdensome, especially when supporting multiple target devices. Taras Sereda led research to evaluate if frontier models could automatically generate low-level kernels from pure PyTorch. Our technique sped up inference on Apple by 1.87X on average! Check out the details here: https://lnkd.in/g4tdEXEi

89

5 Comments

🆕 The Ultimate Guide to Open-Source RL Libraries for LLMs: 9 Frameworks Compared 🎯 Three primary RL applications for LLMs: 1️⃣ RLHF for human preference alignment 2️⃣ reasoning models for problem-solving 3️⃣ multi-turn agentic interactions [via Philipp Moritz of Anyscale ] https://lnkd.in/gcF2dNGq

57

1 Comment

I just published Vol. 143 of "Top Information Retrieval Papers of the Week" on Substack. My Substack newsletter features the 7-10 most notable research papers on information retrieval (including recommender systems, search & ranking, etc.) from each week, with a brief summary, and links to the paper/codebase. This week’s newsletter highlights the following research work: 📚 Production-Scale LLM Ranking for Semantic Search, from LinkedIn 📚 The Role of Embedding Magnitude in Contrastive Learning, from NAIST 📚 Bidirectional Diffusion Models for Dense Text Embeddings, from Perplexity AI 📚 Exposing Reproducibility Failures and Conceptual Flaws in Diffusion Recommenders, from Benigni et al. 📚 Learning Sparse High-Dimensional Embeddings for Efficient Collaborative Filtering, from Vančura et al. 📚 LLM-Driven Autonomous Optimization for Industrial-Scale Recommendation Systems, from Google 📚 Compressing User Histories into Learnable Memory for Scalable Generative Recommendation, from Tencent 📚 Scaling Industrial Ranking Models to 15 Billion Parameters with TokenMixer-Large, from ByteDance 📚 A Reasoning-Enhanced LLM Framework for Recommendation Re-ranking, from Meta 📚 Establishing Scaling Laws for Massive-Scale Recommender Systems, from Meta #InformationRetrieval #ResearchPapers #CuratedContent #Newsletter  #substack

61

2 Comments

Tom Aarsen recently came across two interesting Sentence Transformer models - one finetuned from ModernBERT-base, the other from Pinecone ConstBERT, which surprised me a bit since ConstBERT is typically used as a multi-vector model. Even more surprising? On a simple yet telling benchmark (TripletEvaluator: % of times sim(query, positive) > sim(query, negative)), the ConstBERT-based model slightly edged out the ModernBERT one: 🔹 ModernBERT: 98.8% 🔹 ConstBERT: 99.1% It's a small margin, but fascinating that a model designed for late interaction is holding its own—even excelling—in single-vector settings. Makes you wonder about architectural flexibility and how far you can stretch these models with the right fine-tuning. Curious if others have seen similar results or explored hybrid use cases this way—happy to chat! #NLP #InformationRetrieval #SentenceTransformers #BERT #IR #ML

101

6 Comments