Barathwaj Anandan

Recently, I've been nerd-sniped by the field of reinforcement learning. My current project: building a deep learning model to play Yahtzee using policy gradient RL. I specifically chose Yahtzee over other games, robotic tasks, and LLM workflows because it allowed me to bypass complexity and workload in the following ways: 📌 No Dataset Required: Data is generated on-the-fly during training, eliminating the significant overhead of dataset curation. 📌 Simple Simulation Environment: Allowed me to quickly build a working environment without extensive infrastructure or setup. 📌 Rapid and Affordable Iteration: Policy training runs free and fast on my personal computer and a tiny model is all thats necessary for Yahtzee 📌 Clear Reward Structure: Straightforward rewards (game score) makes it easier to focus directly on the RL algorithm itself. 📌 Personal Motivation: I've never been good at Yahtzee—I always lost as a kid—so there's extra satisfaction in tackling it. This streamlined approach helped me focus on policy gradient methods, particularly appealing due to their importance in fine-tuning LLMs—another area I've recently been exploring. When you scope down problems, they get less intimidating! I'll be sharing updates here and on my upcoming blog. If you're also diving into RL or want to chat about it, feel free to reach out! #ReinforcementLearning #MachineLearning #AI #LearningThroughWriting #LearnByDoing

104

11 Comments

Exciting things at LeRobot! 🤖 We’ve just integrated Meta-World: a benchmark for testing multi-task and generalization abilities in robotic manipulation: https://lnkd.in/d8PGUTUY We’ve also cleaned up our environments and standardized on Gymnasium ≥ 1.x.x and MuJoCo ≥ 3.0.0 across the board. Train your policy with the Meta-World dataset → https://lnkd.in/dHMhST4P All in LeRobot! next is making environments as easy to load as datasets or models 🚀

99

2 Comments

I had a great time presenting our work "Superfast Configuration-Space Convex Set Computation on GPUs for Online Motion Planning" last weekend at #RSS2025 🤓 We propose a method for generating sequences of overlapping convex sets in configuration space in real-time for motion planning. The proposed set-construction algorithm inflates collision-free piece-wise linear paths to sequences of convex sets and is massively parallel. We leverage this to implement it entirely in CUDA. The resulting fast computation times enable the use decomposition-based motion planners that can produce smooth minimum-time trajectories in changing environments! Big thanks to my collaborators Richard Cheng, Tom Stewart, Russ Tedrake, and Daniela Rus for making this happen! Website 🌐 https://lnkd.in/ebw474ru Paper 📚 https://lnkd.in/exuU_Cqr Code 🤖 https://lnkd.in/e98UrGwV P.S. Make sure to check out the python code examples 😁

391

7 Comments

I've been spending the past few weekends building an RL framework for high-speed drone racing — and it's finally ready to share! 🏁🚁 Isaac Drone Racer is a reinforcement learning framework built on top of NVIDIA Robotics IsaacLab. The trained policy hits speeds of 80 km/h and experiences up to 7.5 Gs in tight turns — all learned in just a few minutes using massively parallel simulation. 🔗 GitHub: https://lnkd.in/eZBVWHt5 📹 Demo: https://lnkd.in/e8qdvFZW Next step: building a real drone and tackling the sim2real gap 👀 Big credit to the amazing research coming out of the Robotic Systems Lab of ETH Zürich, the Robotics and Perception Group of University of Zurich and the MAVLab of Delft University of Technology— this project builds on their work. Open to feedback, thoughts, and ideas for improvement — always happy to chat! #ReinforcementLearning #Robotics #IsaacSim #UAV

815

22 Comments

Some signs that Waymo has jumped on the end-to-end machine learning bandwagon. Junko Yoshida has a scoop from attending The Autonomous: signs that Waymo is doing tentative deployments of end-to-end machine learning in their fleet. Complete with a quote from Waymo that seems to support this (or at least does not deny it). Such a change has deep implications for scalability as well as safety. More about this in the article. I fully expect that Waymo will be smarter than a simplistic pixels-in/driving-out approach, as indicated in the story. But what that will look like is speculation at this point. https://lnkd.in/e4vTmRTs

88

9 Comments

🚀 Introducing BroRL: Scaling Reinforcement Learning via Broadened Exploration How to scale RL with compute? We introduce a new dimension of RL Scaling Law: 𝐬𝐜𝐚𝐥𝐞 𝐫𝐨𝐥𝐥𝐨𝐮𝐭𝐬, 𝐧𝐨𝐭 𝐬𝐭𝐞𝐩𝐬. Instead of extending training depth, BroRL increases exploration breadth: scaling N into the hundreds. This simple yet powerful shift delivers continuous, monotonic improvements where step-scaling stops. BroRL (Broad Reinforcement Learning) revives models that saturated after 3K steps in our old ProRL work, setting new SOTA for 1.5B models across a wide range of benchmarks. 🔬 Key insights Theory: Increasing rollout size N reduces the unsampled coupling term, ensuring sampled tokens always contribute positively to correct-mass growth. Simulation: Large-N rollouts eliminate knowledge shrinkage, stabilizing updates and improving pass@k. Empirics: A 1.5B model that had plateaued after 3K ProRL steps revives and keeps improving — reaching Math 63.03, Code 54.20, Reasoning Gym 63.09 under matched compute. Efficiency: Dynamic Sampling pass rate rises from 41% → 62%, and throughput nearly doubles (36.5 → 72.4 samples/s). ⚙️ Takeaway BroRL scales RL along a complementary dimension — broadening exploration rather than prolonging training. It’s a more data- and compute-efficient path toward stable, reliable progress for reasoning and agent models. We’re now integrating ProRL/BroRL into NeMo-RL to power the next generation of reasoning and agentic training. Check our Paper: https://lnkd.in/g-qzFgma Thanks to contributors Jian Hu Mingjie Liu Ximing Lu Fang Wu Zaid Harchaoui SHIZHE DIAO Yejin Choi Pavlo Molchanov Jun Yang Jan Kautz Yi Dong #AI #ReinforcementLearning #LLM #Reasoning #Exploration #NVIDIA #Research #MachineLearning

106

1 Comment

Today marks another step toward making Reinforcement Learning (RL) faster, simpler, and scalable for builders harnessing the Essential Cloud for AI. Our support on the development of torchforge - a new PyTorch-native framework - is helping researchers take large-scale RL workloads from experimentation to production. In partnership with Meta and Stanford University, torchforge was validated on CoreWeave’s Cloud Platform proving stable, efficient, and highly performant – ready to power the next generation of RL workloads. torchforge is now supported on SUNK. More in our recent blog: https://hubs.la/Q03PLJC10

136

𝗘𝘃𝗲𝗿𝘆𝗼𝗻𝗲 𝗵𝗮𝘀 𝗮 𝗰𝗼𝗽𝗶𝗹𝗼𝘁 𝗻𝗼𝘄 — 𝗯𝘂𝘁 𝘁𝗵𝗶𝘀 𝗼𝗻𝗲 𝘀𝗽𝗲𝗮𝗸𝘀 𝗠𝗔𝗧𝗟𝗔𝗕. MATLAB Copilot is 𝑦𝑜𝑢𝑟 AI assistant for modeling, coding, debugging, and teaching — all right inside the MATLAB desktop. It knows your workflow, understands your syntax, and doesn’t hallucinate math. 𝗪𝗮𝗻𝘁 𝘁𝗼 𝘀𝗲𝗲 𝗶𝘁 𝗶𝗻 𝗮𝗰𝘁𝗶𝗼𝗻? 𝗧𝘄𝗼 𝗳𝗿𝗲𝗲 𝘄𝗲𝗯𝗶𝗻𝗮𝗿𝘀 𝗮𝗿𝗲 𝗰𝗼𝗺𝗶𝗻𝗴 𝘂𝗽: 𝐅𝐨𝐫 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐫𝐬 𝐚𝐧𝐝 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐬: Sept 18 – 𝐼𝑛𝑡𝑟𝑜𝑑𝑢𝑐𝑖𝑛𝑔 𝑀𝐴𝑇𝐿𝐴𝐵 𝐶𝑜𝑝𝑖𝑙𝑜𝑡 with Seth DeLand → Turn prompts into code → Explain errors, generate tests → Work faster with grounded suggestions 🔗 Register: https://lnkd.in/gfGJmrTr 𝐅𝐨𝐫 𝐮𝐧𝐢𝐯𝐞𝐫𝐬𝐢𝐭𝐲 𝐟𝐚𝐜𝐮𝐥𝐭𝐲 𝐚𝐧𝐝 𝐢𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐨𝐫𝐬 𝐢𝐧 𝐒𝐓𝐄𝐌 𝐝𝐢𝐬𝐜𝐢𝐩𝐥𝐢𝐧𝐞𝐬: Sept 25 – 𝑇𝑒𝑎𝑐ℎ𝑖𝑛𝑔 & 𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝑤𝑖𝑡ℎ 𝐺𝑒𝑛𝐴𝐼 𝑎𝑛𝑑 𝑀𝐴𝑇𝐿𝐴𝐵 with William Greenwood and Jeff Alderson. → Support student learning → Integrate Copilot with MATLAB Grader → Disable AI during assessments 🔗 Register: https://lnkd.in/gbR_HyjX I’ve tried #MATLAB Copilot — it’s fast, focused, and genuinely helpful. Curious to try it? Or already using it? Let’s chat below. #MATLABCopilot #GenerativeAI #AIforEngineers #AIforEducation #STEM

53

2 Comments

🚨Paper Alert 🚨 ➡️Paper Title: SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation 🌟Few pointers from the paper 🎯Robotic manipulation systems operating in diverse, dynamic environments must exhibit three critical abilities: multitask interaction, generalization to unseen scenarios, and spatial memory. 🎯While significant progress has been made in robotic manipulation, existing approaches often fall short in generalization to complex environmental variations and addressing memory-dependent tasks. 🎯To bridge this gap, authors of this paper introduced “SAM2Act”, a multi-view robotic transformer-based policy that leverages multi-resolution upsampling with visual representations from large-scale foundation model. 🎯SAM2Act achieves a state-of-the-art average success rate of 86.8% across 18 tasks in the RLBench benchmark, and demonstrates robust generalization on The Colosseum benchmark, with only a 4.3% performance gap under diverse environmental perturbations. 🎯Building on this foundation, they have  proposed “SAM2Act+”, a memory-based architecture inspired by SAM2, which incorporates a memory bank, an encoder, and an attention mechanism to enhance spatial memory. 🎯To address the need for evaluating memory-dependent tasks, they have also introduced MemoryBench, a novel benchmark designed to assess spatial memory and action recall in robotic manipulation. 🎯 SAM2Act+ achieves competitive performance on MemoryBench, significantly outperforming existing approaches and pushing the boundaries of memory-enabled robotic systems. 🏢Organization: University of Washington, Universidad Católica San Pablo, NVIDIA, Ai2 🧙Paper Authors: Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru (Helen) Wang, Dieter Fox, Ranjay Krishna, Jiafei Duan 📝 Read the Full Paper here: https://lnkd.in/gn5VMjBn 🗂️ Project Page: https://sam2act.github.io/ 🎥 Be sure to watch the attached Demo Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

112

3 Comments

🎯 AI Execution Stack (JAX vs PyTorch 2.x): From Model to Machine Code Earlier this week, I attended the 2025 JAX & OpenXLA DevLabs — and it was incredibly insightful. The deep dives into JAX’s lowering pipeline, StableHLO, and the broader OpenXLA ecosystem inspired me to visualize the full AI execution stack. Comparing JAX/XLA with the PyTorch ecosystem helped me better understand the low-level architecture of ML systems, including core concepts like IR (Intermediate Representation), ML Compiler, and Runtime Execution. 🔍 This visualization covers: 🔹 JAX → JAXPR → XLA → HLO → TPU/GPU/CPU 🔹 PyTorch 2.x → FX → Inductor → Triton/NvFuser/C++ → GPU/CPU 🔹 PyTorch → ONNX → TensorRT / ONNX Runtime → GPU It’s fascinating to see how ML compilation is evolving toward modular, backend-agnostic design, enabling portable and efficient execution across diverse hardware. 🙏 Special thanks to Han Qi (PyTorch/XLA expert) for generously sharing insights and helping clarify the internals of the stack. Also grateful to my teammates for the ongoing technical discussions and encouragement. 💬 Feel free to share feedback or correct anything in the diagram — I’m still learning too! #JAX #XLA #StableHLO #HLO #PyTorch #TorchInductor #ONNX #TensorRT #AIInfrastructure #MachineLearning #DeepLearning

375

11 Comments

🚀 Advances in Evaluating Large Language Models – April 2025 Overview As LLMs become central to enterprise, healthcare, and research applications, rigorous evaluation is no longer optional—it's foundational. Pun Intended !! :) In my latest piece, I dive into the evolving science and engineering of LLM evaluation. I synthesize recent breakthroughs across: 🔍 Lifecycle observability & instrumentation ⚠️ Error analysis & failure mode detection 🧠 Human-in-the-loop evaluation 📊 Cost-performance trade-offs 📚 And more—complete with links to must-read research papers and abstracts Whether you're building, deploying, or governing LLM systems, this article offers a roadmap to think more critically about performance, safety, and reliability. 📝 Read the full article here: https://lnkd.in/gWzc6iKk Let’s shape a future where language models are not just powerful—but also transparent, accountable, and aligned with human values. #AI #LLM #MachineLearning #ResponsibleAI #LLMEvaluation #AIGovernance #Observability #ErrorAnalysis #AIResearch 

41

2 Comments

We built ProfBench to raise the bar for LLMs - literally. At NVIDIA, we worked with domain experts to create a benchmark that goes far beyond trivia and short answers. ProfBench tests LLMs on complex, multi-step tasks that demand the kind of reasoning, synthesis, and clarity you'd expect from a PhD physicist or MBA consultant. 🌎 This isn’t just a dataset drop. It’s a global collaboration: 38 professionals across 8 countries contributed over 7,000 expert-written rubrics across finance MBA 💵, consulting MBA 📊, chemistry PhD 🧪and physics PhD 🚀. 🧗Every prompt and grading rubric was handcrafted, requiring tens of hours of dedicated and focussed work. Now fully supported in the NeMo Evaluator SDK, ProfBench enables reproducible, rubric-based evaluations and side-by-side model comparisons. 🔗 ProfBench on Hugging Face https://lnkd.in/g2mCMcnc 🔗 NeMo Evaluator SDK https://lnkd.in/gF6SQCwt I’m so proud of the team that made this happen. Let’s keep pushing what AI can do. #ProfBench #LLM #AIevaluation #NeMo #NVIDIA #OpenSourceAI #AIresearch #AgenticAI #GenerativeAI #BuiltByExperts #GTCDC Work done with Jaehun Jung Ximing Lu Shizhe Diao Ellie Evans Jiaqi Zeng Pavlo Molchanov Yejin Choi Jan Kautz Yi Dong Collaborators: Vivienne Zhang Isabel Hulseman, MBA Seph Mard Pablo Ribalta, PhD Grzegorz Chlebus Wojciech Prazuch Ankit J. Patel

200

3 Comments

Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos https://lnkd.in/gsqxp6tz The paper introduces Sa2VA, a framework that combines SAM2 (Segment Anything Model 2) with LLaVA (Large Language and Vision Assistant) to achieve dense grounded understanding of images and videos. By integrating SAM2’s powerful segmentation capabilities with LLaVA’s language-driven image understanding, Sa2VA enables detailed and context-aware interpretations of visual content. The approach is designed for tasks requiring fine-grained spatial reasoning and multimodal understanding, such as object localization, scene understanding, and video analysis. This fusion enhances the precision and flexibility of grounded visual understanding, paving the way for advanced applications in vision-language tasks.

32