Pavel Nesterov

DeepSeek just published a wild paper that makes long-context LLMs 10x faster. Not only does it cut costs and boost inference speed, but it also beats full attention models. Here's everything you need to know: Long-context models can change everything—handling full books, giant codebases, and deep reasoning. But standard attention scales poorly and large sequences become slow and expensive to process. DeepSeek's NSA (Native Sparse Attention) changes that. How does it work? NSA blends three techniques: ▸ Dynamic hierarchical sparsity – skips redundant computations ▸ Coarse-grained token compression – groups less important tokens ▸ Fine-grained token selection – keeps critical details sharp It speeds up attention by compressing less important tokens, selecting only the most relevant ones for precise attention, and using a sliding window to retain local context. This reduces unnecessary computations but preserves long-range relationships. How fast is it? NSA crushes full attention in efficiency: ▸ 11.6x speedup in decoding ▸ 9.0× speedup in forward propagation ▸ 6.0× speedup in backward propagation How accurate is it? NSA outperforms Full Attention on 7 out of 9 benchmarks, achieving higher overall performance. It also shows big improvements in reasoning tasks, with a +4.2% gain in complex reasoning and +3.4% in problem-solving Most sparse attention methods only optimize for inference. NSA is trainable by nature, meaning it slashes pretraining costs while keeping model quality high. It’s also built for real-world performance to maximize GPU efficiency—making it even faster on modern hardware. Read the paper: https://lnkd.in/gNTYCUGx ↓ Are you an AI developer? Check out https://AlphaSignal.ai to get a daily summary of breakthrough models, repos and papers in AI. Read by 200,000+ devs.

1,075

36 Comments

The complexity of PPO leads practitioners to avoid online RL in favor of RL-free or offline algorithms (e.g., DPO), but why not just use simpler versions of online RL? TL;DR: REINFORCE and RLOO have been shown to work well for training LLMs. And, they do not require a value model, which drastically reduces cost and complexity. Therefore, instead of completely avoiding online RL due to the overhead associated with PPO, we can just use algorithms that provide the benefits of online RL without the unnecessary complexity. Policy gradient: RL optimizers operate by computing the gradient of the RL objective (maximizing rewards) w.r.t. policy parameters—this is called the policy gradient—and performing gradient ascent. A basic policy gradient is easy to compute (see figure) but has high variance. So, we introduce some extra complexity to get stable gradients / training. Actor-critic: We can reduce variance by incorporating the advantage function (instead of the raw reward for a trajectory) into the policy gradient estimate. When we do this, the value function acts as a normalizer / baseline to the reward estimate for a state. PPO estimates advantage with an additional copy of the LLM—called a value model or critic—that is trained via an MSE loss to predict the final reward for every token in a sequence. The critic is on-policy (i.e., it depends on the current parameters of the policy during training) and is trained alongside the policy itself during RL. This is called an actor-critic framework. REINFORCE: Due to keeping and extra copy of the LLM in memory that is trained, actor-critic frameworks come with additional cost and complexity. As an alternative, we can use the much simpler REINFORCE algorithm, which estimates the value function via an average of rewards (either a moving average of all rewards or an average of rewards in a batch) throughout training. Estimating the value in this way is low cost and removes the need for a critic. REINFORCE does yield higher variance relative to actor-critic methods, but recent work has shown that this does not matter in the context of LLMs. Algorithms like PPO were developed in a prior generation of DeepRL research, where neural networks were primarily trained from scratch. In the LLM domain, we are finetuning pretrained models with a strong prior—unstable gradients are much less of a worry. For this reason, even simple online RL algorithms like REINFORCE work well for training LLMs despite their higher variance. RLOO: To reduce variance relative to REINFORCE, we can also use REINFORCE leave one out (RLOO). This algorithm reduces variance by sampling multiple completions for every prompt and estimating the value function as the average of rewards for other completions—excluding the completion for which the value is being estimated—to the same prompt.

230

6 Comments

xLSTM for Automated Stock Trading: https://lnkd.in/dyKp6QGE "xLSTM-based model outperforms LSTM-based methods in key trading evaluation metrics, including cumulative return, average profitability per trade, maximum earning rate, maximum pullback, and Sharpe ratio. These findings mark the potential of xLSTM for enhancing DRL-based stock trading systems." In this reinforcement learning approach, xLSTM is used in both actor and critic components and increases the performance.

265

3 Comments

If you would like to get hands-on building AI applications with Hugging Face, check this series of articles I have recently written: 1️⃣ Text-Based Applications Building Chatbot Using HuggingFace Open Source Models https://lnkd.in/duf9-RKA Building Text Translation System using Meta NLLB Open-Source Model https://lnkd.in/dDRjEFhT 2️⃣ Speech-Based Applications Zero-Shot Audio Classification Using HuggingFace CLAP Open-Source Model https://lnkd.in/dMN6Bmxz Building & Deploying a Speech Recognition System Using the Whisper Model & Gradio https://lnkd.in/dajrBiyZ Building Text-to-Speech Systems Using VITS & ArTST Models https://lnkd.in/dUKqxwST 3️⃣ Image Based Applications Step-by-Step Guide to Zero-Shot Image Classification using CLIP Model https://lnkd.in/di6eqkfV Building an Object Detection Assitant Application: A Step-by-Step Guide https://lnkd.in/dKQuedUC Zero-Shot Image Segmentation using Segment Anything Model (SAM) https://lnkd.in/dzDY_-Gq Building Zero Shot Depth Estimation Application Using DPT Model & Gradio https://lnkd.in/dPssdC94 4️⃣ Vision Language Applications Building Visual Questioning Answering System Using Hugging Face Open-Source Models https://lnkd.in/dMPchJDk Building Image Captioning System using SalesForce Blip Model https://lnkd.in/dcKpM_3z Building Image-to-Text Matching System Using Hugging Face Open-Source Models https://lnkd.in/dwAfNw_k 5️⃣ Application Deployment Deploying Machine Learning Models on Hugging Face Hub with Gradio Integration https://lnkd.in/d6iQMEm6 ➡️ You can find the articles and the codes for each article in this GitHub repo: https://lnkd.in/dG5jfBwE ➡ Do not miss the 50% discount on To Data & Beyond yearly subscription: https://lnkd.in/dZ2Sjkhk ➡ Check out my new ebook: Efficient Python for Data Scientists: https://lnkd.in/dMs_53Wz

305

6 Comments

MLM: Learning Multi-task Loco-Manipulation Whole-Body Control for Quadruped Robot with Arm Arxiv: https://lnkd.in/e5cAEfbG Video: https://lnkd.in/epEcFPhM Can quadrupeds seamlessly blend locomotion and manipulation—walking, balancing, and handling objects—all in one unified framework? MLM introduces a whole-body, multi-task control approach that trains quadruped robots with arms to perform diverse loco-manipulation tasks in a single end-to-end policy, outperforming modular baselines and enabling fluid, human-like coordination. 🔁 At a Glance 💡 Goal: Learn generalizable loco-manipulation policies for quadruped robots with arms, unifying locomotion and manipulation within one control policy. ⚙️ Approach: Whole-body RL policy: Jointly controls legs + arm in a unified action space. Multi-task training: Shared representation across locomotion (walking, turning, balancing) and manipulation (pushing, picking, placing). Domain randomization & sim-to-real transfer for robustness. 📈 Impact (Key Metrics) 🧪 Simulation (Isaac Gym) Outperforms hierarchical and modular baselines by +20–30% success rate in complex loco-manip tasks. Learns smooth transitions between tasks (e.g., walking → pick-up → carry). 🤖 Real-World (Unitree Go1 + 4-DoF Arm) Successfully performs pick-and-carry and push-and-move tasks with high stability. Demonstrates zero-shot adaptation to new terrains and object placements. 🔬 Experiments 🧪 Datasets / Envs: Multi-task simulated environments (navigation, grasping, pushing) 🎯 Tasks: Walk + manipulate, push while balancing, pick-and-carry, obstacle negotiation 🦾 Robot: Unitree Go1 quadruped with 4-DoF arm 📐 Input: Proprioception + onboard RGB-D for task perception 🛠 How to Implement 1️⃣ Unified Policy Learning Train end-to-end RL policy with legs and arm sharing observation-action space. 2️⃣ Multi-task Curriculum Gradually increase task complexity (navigation → manipulation → combined loco-mani). 3️⃣ Sim-to-Real Transfer Apply domain randomization on physics, textures, and sensor noise for robustness. 📦 Deployment Benefits ✅ Unified whole-body control—no hand-crafted locomotion/manipulation modules ✅ Scales across many loco-mani tasks with one policy ✅ Robust to terrain, obstacles, and object variability ✅ Brings quadrupeds closer to true generalist field robots Takeaway MLM marks a step towards general-purpose quadrupeds capable of both moving through the world and shaping it. By unifying loco-manipulation, it pushes quadrupeds beyond mobility into multi-task embodied intelligence. Follow me to know more about AI, ML and Robotics!

76

📢 New preprint: TCP: A Benchmark for Temporal Constraint-Based Planning 📄 arXiv: https://lnkd.in/dt3Ap_4F 📊 Dataset: https://lnkd.in/dfYJVdPJ We introduce 🧠 TCP, a benchmark that systematically evaluates large language models (LLMs) on planning under interdependent temporal constraints, where the goal is to compute the optimal schedule from naturalistic dialogue. Unlike prior work that focuses on isolated temporal reasoning, TCP features two categories of planning problems: 📆 Short-horizon tasks: Within a single day (involving clock-time and time zone reasoning) 📅 Long-horizon tasks: Up to one week (requiring date and weekday computations) Each instance embeds asynchronous task execution, individual availability patterns, and temporal dependencies within a collaborative dialogue—posing a rich challenge for temporal reasoning. 🔍 Key findings: 🧩 Asynchronous planning is still challenging for current models. 🌐 Models frequently fail to handle time zone alignment correctly. 🌍 We evaluate across 10 diverse domains, revealing domain- and constraint-specific model weaknesses. 📉 Even Gemini 2.5 Pro, one of the most capable reasoning LLMs, achieves only 71.4% accuracy, underscoring the benchmark’s difficulty. 🧠 Reasoning models consistently outperform standard LLMs by producing more structured, globally coherent, and reflective outputs, which are crucial for handling multiple interacting temporal constraints. TCP offers a focused testbed for advancing LLMs’ abilities in temporal reasoning and constraint-based planning. 👥 Shout out to my collaborators Sikuan Yan, Moy Yuan, Xianglong Hu, Fangru Lin, and my best advisor Andreas Vlachos. #LLM #TemporalReasoning #Planning #Benchmark #NLP #AI

46

Just found one of the most compact and insightful resources for learning Transformers & Large Language Models (LLMs)! Stanford University's CME 295 course released a 𝐕𝐈𝐏 𝐂𝐡𝐞𝐚𝐭𝐬𝐡𝐞𝐞𝐭 that gives a visual, easy-to-digest overview of key LLM concepts, including: 🧩 Tokens, tokenization types, and embeddings 🎯 Attention mechanism & Multi-Head Attention 🏗️ Transformer architecture explained with diagrams Whether you're just starting with LLMs or practicing more the foundations, this cheat sheet is gold. Created by Afshine Amidi and Shervine Amidi! 💯 Free and beautifully illustrated. A must-have for anyone working with LLMs, RAG, or Transformers. If you enjoy learning about AI systems and building with LLMs, follow me for more resources like this! #AI #LLMs #Transformers #MachineLearning #DeepLearning #AIEngineering #RAG #NLP #OpenSource

52

2 Comments

GRPO, simplified. Training reasoning-capable LLMs needs GPU, a lot of it. But a new method from DeepSeek—Group Relative Policy Optimization (GRPO)—is changing the game. Unlike PPO-based RLHF that requires four large models and massive compute, GRPO reduces the model count, avoids separate reward/value networks, and enables training on a single GPU—even for complex reasoning tasks. Link: https://lnkd.in/gycxZjej

52

3 Comments

DeepSeek recently published a paper on "Native Sparse Attention". Here is why it matters. The attention mechanism is core to the transformer architecture, which in turn is a building block for today's LLMs. Unlike recurrent networks, the attention mechanism can find causal relationships even in long inputs. Simply put, if you feed a 10,000 word book to a recurrent network, it is heavily compressed. Some of the causal relationships in input are lost. Example: Why did Harry Potter get his scar? This long range dependency could be lost. The attention mechanism is able to avoid this. It takes the entire input sequence together, and computes their relationships together, conserving long-range dependencies. DeepSeek has just published how they sped up this mechanism, using GPU-specific optimizations. Instead of computing attention in a dense manner (all-to-all O(N²) time complexity), DeepSeek uses SPARSE attention. Think of this as a sliding window of blocks, reducing time complexity to O(N*W). The unique contribution of this paper is the native support for this kind of sparse attention compute. DeepSeek is going down to the level of hardware optimisations to speed up training. Let me know your thoughts around this. Cheers! #AI #LLMs #DeepSeek

133

2 Comments

Using Structured Output and experiencing hallucinations? Get a 100% bulletproof solution that eliminates them. 𝗣𝗿𝗼 𝘁𝗶𝗽: 𝗶𝗻𝗷𝗲𝗰𝘁 𝗿𝗶𝗰𝗵𝗲𝗿 𝗶𝗻𝘁𝗲𝗻𝘁 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗶𝗻𝘁𝗼 𝘆𝗼𝘂𝗿 𝘀𝗰𝗵𝗲𝗺𝗮 Use Pydantic description on each field to tell the LLM what “good” looks like - beyond just str/int. Treat it like a field-level system prompt to control formatting, constraints, tone, and edge cases. 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗵𝗮𝗽𝗽𝗲𝗻𝘀 𝗯𝗲𝗵𝗶𝗻𝗱 𝘁𝗵𝗲 𝘀𝗰𝗲𝗻𝗲𝘀  • OpenAI uses 𝗖𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 when you opt into Structured Output.  • Your Pydantic model is converted into an LLM-readable text via .𝘮𝘰𝘥𝘦𝘭_𝘫𝘴𝘰𝘯_𝘴𝘤𝘩𝘦𝘮𝘢().  • The model is guided to produce content that matches your schema, field by field - instead of free-form text. 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗸𝗶𝗹𝗹𝘀 𝗵𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗶𝗼𝗻𝘀  • Constrained Decoding = the model can only emit tokens that keep the output valid against your schema.  • Descriptive fields = fewer ambiguous generations, tighter adherence to your domain.  • Explicit ranges & enums = no impossible values sneak in. 𝗠𝗶𝗻𝗶-𝗰𝗵𝗲𝗰𝗸𝗹𝗶𝘀𝘁  • Mirror your business object in Pydantic  • Add a specific description to every field  • Use enums/regex/ge/le bounds  • Validate server-side after generation Want the entire cheatsheet I use to reduce hallucinations to near-zero? Comment “𝗰𝗵𝗲𝗮𝘁𝘀𝗵𝗲𝗲𝘁” and I’ll share it.

36

21 Comments

New Blog, Agentic AI 🎉 https://lnkd.in/diEkbC83 In this article, I have highlighted: 1. Approaches to formulating problems with agentic thinking 2. The distinction between static workflows and agentic AI 3. How LangChain simplifies glue code, yet faces limitations in control flow 4. LangGraph’s orchestration capabilities with flexibility, state management, memory and the query 5. A practical case study on road scene analysis using an agentic flow #agenticai #agentic #agents #LangGraph #LangChain

28

1 Comment

Here's how to write a fast MXFP4/NVFP4 dequant GEMV kernel for batch-size=1: -Use a mapping + tl.gather to map the quant matrix block indices into double the fp4 range -Load the scales as e8m0.view(uint8) and use tl.exp2 to convert ~13% faster than SPLIT-K with tl dot_scaled

60

4 Comments

🚀 Exciting Update! 🚀 We just added demo visualizations to our paper "MeshSplats: Mesh-Based Rendering with Gaussian Splatting Initialization" 🎥✨ https://lnkd.in/dxD_cm4P https://lnkd.in/dCdrFpAP Gaussian Splatting (GS) has revolutionized 3D rendering, but it often bypasses traditional ray tracing methods that provide advanced lighting effects like shadows and reflections. With MeshSplats, we bridge this gap by transforming GS into a mesh-like format, enabling ray tracing-compatible rendering without additional training. 🔹 Key Highlights: ✅ Converts GS to mesh faces for ray tracing ✅ Immediate usability post-transformation ✅ Enhanced reconstruction with dedicated optimization ✅ Expands applications in computer graphics & image processing Check out the visualizations and let us know your thoughts! 💡👇 GMUM - Group of Machine Learning Research

31

🚀 Understanding Transformer Architectures: Encoder-Only, Decoder-Only, and Encoder-Decoder 🌟 Transformers revolutionized NLP with the introduction of the paper "Attention Is All You Need." These architectures form the backbone of many state-of-the-art models and have three primary variations tailored for specific tasks: 1️⃣ Encoder-Only Models: Focused on understanding text, these models are perfect for tasks like sentiment analysis or classification. They emphasize extracting the meaning of the input rather than generating new sequences. 👉 Example: Turning "I’m happy" into a sentiment score of 1. 2️⃣ Decoder-Only Models: Built for sequence generation, decoder-only models take an input and produce creative outputs like text generation or auto-completions. 👉 Example: Transforming "I hope you are" into "Good and healthy." 3️⃣ Encoder-Decoder Models: Designed to transform input into a different output, these models excel in tasks like translation or summarization. 👉 Example: Converting "I’m happy" into "Io sono contento" (Italian translation). Understanding the right architecture for your NLP project is crucial to achieving optimal results. Whether it's analyzing sentiment, generating content, or translating languages, transformers continue to push the boundaries of what's possible in AI. 💡 #AI #MachineLearning #Transformers #NLP #Innovation

21