Fedor Borisyuk

DeepSeek R1’s training method (GRPO) is now fully reproducible—the entire codebase is on GitHub. GRPO takes a new approach to RL for LLMs: instead of using a separate reward model, it estimates rewards dynamically from a group of outputs. No labeled data, no human preference models—just pure RL, making training more efficient and scalable. Huge kudos to Deepseek for describing it in their paper. This is the power of #opensource. Billions of investments in the last years, thousands of research papers, all compressed in a few hundred lines of code. This is likely the Sputnik of AI. 🔗 Github link: https://lnkd.in/d4DzY_AH #AI #OpenSource #DeepSeekR1 #ReinforcementLearning #MachineLearning H-FARM AI H-FARM

576

21 Comments

DeepSeek recently published a paper on "Native Sparse Attention". Here is why it matters. The attention mechanism is core to the transformer architecture, which in turn is a building block for today's LLMs. Unlike recurrent networks, the attention mechanism can find causal relationships even in long inputs. Simply put, if you feed a 10,000 word book to a recurrent network, it is heavily compressed. Some of the causal relationships in input are lost. Example: Why did Harry Potter get his scar? This long range dependency could be lost. The attention mechanism is able to avoid this. It takes the entire input sequence together, and computes their relationships together, conserving long-range dependencies. DeepSeek has just published how they sped up this mechanism, using GPU-specific optimizations. Instead of computing attention in a dense manner (all-to-all O(N²) time complexity), DeepSeek uses SPARSE attention. Think of this as a sliding window of blocks, reducing time complexity to O(N*W). The unique contribution of this paper is the native support for this kind of sparse attention compute. DeepSeek is going down to the level of hardware optimisations to speed up training. Let me know your thoughts around this. Cheers! #AI #LLMs #DeepSeek

133

2 Comments

🆕 The Ultimate Guide to Open-Source RL Libraries for LLMs: 9 Frameworks Compared 🎯 Three primary RL applications for LLMs: 1️⃣ RLHF for human preference alignment 2️⃣ reasoning models for problem-solving 3️⃣ multi-turn agentic interactions [via Philipp Moritz of Anyscale ] https://lnkd.in/gcF2dNGq

57

1 Comment

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Let's learn more! In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point—MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable—without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. I created it while taking inspiration from Femke Plantinga's post on this topic. If you want to understand this better with code, I have linked my article in the comments. ____ If you want to learn AI/ML engineering, I have put together a free PDF (530+ pages) with 150+ core DS/ML lessons. Get here: https://lnkd.in/gi6xKmDc ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

1,382

48 Comments

🧠💬 Memory in LLMs A practical guide showing how to implement conversational memory in LLMs using LangGraph, demonstrated through a therapy chatbot. Features code examples for basic retention, trimming, and summarization approaches. Learn to build memory-aware apps 👉 https://lnkd.in/gybcrV5v

968

21 Comments

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

As you start investigating performance of different algorithms/models on different GPUs having an intuition on how Memory is accessed by threads (and groups of threads) can help you to gain a deeper appreciation of lower-level optimizations. This becomes even more complicated as you do multi-gpus and multi-node (groups of multiple GPUs). Read more: https://lnkd.in/eN9nteYx #deeplearning #machinelearning

54

2 Comments

New article: TimeXer for Time Series Forecasting with Exogenous Features TimeXer is a Transformer-based model, specifically built to take into account exogenous features. Thus, it does not only rely on past values of the series, but also on external information that may help forecast our target. This is especially useful with time series that are impacted by external factors. For example, the demand for a product can change if there is a holiday, or the demand for electricity is impacted by the temperature outside. In this article, we discover the architecture and inner workings of TimeXer and apply it in a small forecasting project using the available implementation in neuralforecast from Nixtla. Enjoy the read! #timeseries #forecasting #datascience #machinelearning #deeplearning #artificialintelligence #python #programming https://lnkd.in/eymfjmh8

90

4 Comments

Introducing Generative Adversarial Distillation (GAD): a novel GAN-style formulation and framework that facilitates both on-policy and black-box distillation of large language models (LLMs). GAD is the first technique to enable block-box on-policy distillation from proprietary teachers where internal logits or parameters are inaccessible, or distillation between teacher and student LLMs with incompatible vocabularies. GAD expands our prior work on white-box on-policy distillation (i.e., MiniLLM), pioneering block-box on-policy distillation for LLM training. Specifically, GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM’s, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation.  In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation. Our team has been conducting fundamental research in knowledge distillation with wide adoptions across the industry. -         MiniLM: We introduced multi-head attention distillation, establishing the most effective distillation method for BERT-style models. The open-source MiniLM models (e.g., 6x384) have become the most widely utilized small encoder models on the Hugging Face. -         MiniLLM: Our proposed Reverse KLD is recognized as one of the most effective, de facto on-policy distillation approaches for modern LLM training, which has been widely used by Thinking Machines, Gemma, and many other teams and models. -         BitDistill: We proposed BitNet Distillation to finetune off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (ternary weights {-1, 0, 1}), achieving performance parity with the full-precision counterparts on specific downstream tasks. -         GAD: The development of Generative Adversarial Distillation (GAD) now allows for black-box on-policy distillation, overcoming two major prior limitations: (1) Distillation from proprietary teachers where internal logits or parameters are inaccessible; (2) Distillation between teacher and student LLMs with incompatible vocabularies. https://lnkd.in/gMaP2c7w

125

2 Comments

🚀 Big news — NeMo Evaluator is now open source! 🧪 We’re thrilled to announce that NeMo Evaluator SDK — the engine behind NVIDIA’s internal model benchmarking — is now open sourced and available to everyone: 🔗 https://lnkd.in/e3yaznR4 NeMo Evaluator SDK is an open-source platform for robust, reproducible, and scalable evaluation of large language models. It lets you run hundreds of benchmarks across popular evaluation harnesses against any OpenAI-compatible model API, all from auditable Docker containers for trustworthy and reproducible results. NeMo Evaluator is built on four core principles: ✅ Reproducibility by Default: Every config, random seed, and dependency is captured automatically for repeatable results. ⚙️ Scale Anywhere: Run on your laptop, a Slurm cluster, or cloud backends like Lepton AI without changing your workflow. 📊 State-of-the-Art Benchmarking: Access 100+ benchmarks from 18 open-source harnesses, covering reasoning, math, code, safety, and more. 🧩 Extensible and Customizable: Add new benchmarks, integrate private datasets, or connect custom exporters to your existing MLOps tooling. This release is a major step toward transparent, standardized, and community-driven model evaluation. From here on, we’ll be building NeMo Evaluator in the open — and we’d love for you to join us: try it, break it, file issues, contribute, and help shape its future! ⭐️ Explore it here → https://lnkd.in/e3yaznR4 If you find it useful, don’t forget to star the repo and help us grow the community! #NVIDIA #NeMo #Evaluator #OpenSource #LLM #Evaluation #AIResearch #MLOps #Benchmarking

255

3 Comments

One of the persistent challenges in using large language models (LLMs) is getting them to follow instructions reliably — especially when the instructions are subtle or domain-specific. DeepMind’s latest research introduces Symbol Tuning, a simple yet powerful fine-tuning method that significantly improves an LLM’s ability to follow symbolic prompts (e.g., bullet points, XML, Markdown, or code-like instructions) in zero-shot and few-shot settings. https://lnkd.in/gzKDdHQ2 Why this matters: 🔹 Improves instruction following in GPT-class models 🔹 Works with tiny amounts of data (just 100K tokens!) 🔹 Boosts performance in math, code, and reasoning-heavy tasks 🔹 Enhances models' ability to generalize across symbolic formats This has massive implications for building enterprise agents, RAG pipelines, and developer copilots that need high-precision, structured interaction with users or data. A great reminder: sometimes, small, well-targeted innovations create massive gains. #LLM #InContextLearning #SymbolTuning #PromptEngineering #DeepMind #GenAI #AIResearch #InstructionFollowing #EnterpriseAI #DeveloperTools

199

31 Comments

In a new blog by Smallest AI, we talk about issues in large fused kernels, particularly problems with: 1. Dynamic Input Shapes 2. Synchronization and Deadlocks 3. L1 cache latency compounding 4. Cross SM coordination 5. Warp context bottlenecks Further, we segway into what would it take to build a sub 50ms(and lower) speech model. Link and author name in comments, let us know what you think!

134

7 Comments

Speeding up PyTorch performance usually requires the use of intermediate optimization frameworks, or writing low-level optimized kernels. This can be burdensome, especially when supporting multiple target devices. Taras Sereda led research to evaluate if frontier models could automatically generate low-level kernels from pure PyTorch. Our technique sped up inference on Apple by 1.87X on average! Check out the details here: https://lnkd.in/g4tdEXEi

89

5 Comments

More and more LLM models are released in native quantized formats. This blog (https://lnkd.in/gfTcp5yM) provides a brief overview of #LLM #quantization formats and methods, plus insights into the "native quantization" of DeepSeekV3.1 and GPT-OSS.

195

1 Comment

Nvidia Dynamo Distributed Inference Framework: Rust mentioned btw. This project looks awesome as a tool for distributed inference. They’ve put a lot of thought into architecting this, with the ability to provision separate resources for disaggregated prefill and decode, dynamic GPU scheduling based on demand, LLM aware request routing, accelerated data transfer with a new data transfer library called nixl, and smart kv cache offloading. This framework comes in a modular format with a Python front end and a rust backend, so you get safety and fearless concurrency with the ease of Python and the ability to use the whole thing, or only the parts you need. Finally, it supports vLLM as an inference backend (which is my favorite inference library). This project is also fully open-source with an Apache2 license, discord, and an engaging community. I can’t wait to PoC this and share additional feedback! Check out the GitHub here: https://lnkd.in/eJSG9Gby #softwareengineering #gpus

51

🗺️ Monte Carlo Tree Search in LLMs: When AI Plays Chess With Itself Remember how AlphaGo beat humanity at Go? MCTS is that same "try a bunch of moves, see what works" strategy, but now we're teaching ChatGPT to use it. Instead of game moves, we're exploring different reasoning paths. It's like giving your LLM anxiety, but productive anxiety. How MCTS Works in LLMs (The "Choose Your Own Adventure" Method) Start → Explore moves → Simulate outcomes → Pick winner Question → Explore reasoning paths → Evaluate answers → Pick best chain The Four Sacred Steps: 1.   Selection: Start from your question, follow promising thought branches  "Should I solve this math problem algebraically or geometrically?" 2.   Expansion: Add new reasoning steps to explore "Let's try factoring... wait, or maybe substitution?" 3.   Simulation/Rollout: Follow each path to see where it leads "If I factor this, I get... oh no, imaginary numbers" 4.   Backpropagation: Update scores based on what worked "Geometric approach = 10 points, Algebraic = tree fell over" Real Implementation Examples: AlphaCode Style (DeepMind) ·      Generates multiple code solutions ·      Each solution = tree node ·      Tests them against examples ·      Explores variations of promising approaches ·      "What if I use recursion? Dynamic programming? Brute force? ALL THREE?" Tree of Thoughts (ToT) ·      Each thought step becomes a node ·      LLM generates multiple next steps ·      Evaluates each with "is this promising?" ·      Prunes bad branches early ·      "This reasoning is going nowhere, ABORT ABORT" RAP (Reasoning via Planning) ·      Treats reasoning as path-finding ·      World model = problem state ·      Actions = reasoning steps ·      Rewards = correctness scores The Bottom Line: MCTS in LLMs is like giving your model ADHD medication - it can finally focus on exploring options systematically instead of blurting out the first thing that comes to mind. Yes, it's slower. Yes, it's expensive. But when you absolutely need the right answer and have compute to burn, it's the difference between "probably correct" and "definitely correct (we checked 1000 ways)." The Future: Teaching LLMs to recognize when they need MCTS vs. when they can just wing it. Because nobody needs a search tree to answer "How's the weather?" https://lnkd.in/gNifgCGx

24