Reynold Xin

[AI paper] GRPO has a hidden flaw in Multi-Reward settings. It’s time to decouple your normalization. 📉 As AI Architects, we are moving from single-objective RL (just "get the answer right") to multi-objective RL (accuracy + format + length + safety). We typically sum these rewards and throw them into GRPO. A new paper from NVIDIA, "GDPO: Group reward-Decoupled Normalization Policy Optimization," demonstrates that this naive summation causes Reward Signal Collapse. The Architectural Failure Mode: If you sum disparate rewards (e.g., a binary Format reward and a scalar Accuracy reward) and then normalize, different raw reward combinations can map to identical advantage values. Example: A rollout with rewards (0, 2) might yield the same normalized advantage as (0, 1) due to group statistics, effectively deleting the gradient signal for the second objective . The Solution: GDPO The fix is architectural: Decoupled Normalization. Instead of normalizing the sum, GDPO normalizes each reward objective independently within the group before aggregation. Impact: This restores the resolution of the training signal. On Tool Calling and Math Reasoning tasks (DeepSeek-R1/Qwen), GDPO converges where GRPO fails or plateaus. If you are training agents that must balance strict formatting (JSON) with reasoning quality, this is the correct objective function to use. 👇 Link to the paper in the comments. #AIArchitecture #RLHF #NVIDIA #GRPO #DeepLearning #LLM #Alignment #Research

21

1 Comment

KDD 2026 paper +3 🚀 Excited to share that our team had 3 papers accepted to the 2026 KDD! These works represent recent years of our efforts on large-scale recommendation model training, spanning model co-design, GPU kernel optimization, and compiler optimization across the full training stack 🚀 Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design RecCompl: Efficient Model Compilation for Industrial Scale Recommendation Models with PyTorch 2 Optimus: A Generic Operator-Level PyTorch Model Transformation Framework The first work introduces Kunlun, our next-generation scaling-up foundation model architecture for GEM (Generative Ads Model). We recently uploaded the paper to arXiv (https://lnkd.in/gKzFSzcF), which also includes our newly open-sourced GDPA kernel work (https://lnkd.in/gpJ879HG) The second and third works focus on our optimizations on PyTorch 2 compiler stack for large-scale recommendation systems. RecCompl shares how we achieved full-model compilation for production recommendation models with PT2. Optimus is a continuation of our operator-level graph optimization work on PT2. We previously shared some of the core ideas in our PyTorch blog. (https://lnkd.in/gBi3fUqJ) Over the past three years, thanks to the amazing effort from the team, we’ve solved many real-world production challenges and successfully launched these optimizations across most Ads models in production. Huge congrats to the entire team — I’m incredibly proud of what we’ve built together. A lot of hard work, late nights, and iteration went into making these systems work in real production environments 💪 Excited to continue pushing large-scale recommendation systems forward and looking forward to sharing more details soon 🔥

139

2 Comments

For decades, companies chased cheaper labour. Now they’re eliminating it. Oracle’s AI pivot signals the end of traditional labour arbitrage. They moved software engineering to lower-income countries to save money. The logic was simple: cut costs and improve the P&L by shifting roles to cheaper markets. Cheap labour was never the real advantage in tech. It often led to lower-quality software. What matters is great engineers, regardless of location. If AI is writing the code, the value shifts. Not to more engineers, but to better ones. The future is smaller teams of exceptional engineers, amplified by AI. #AI #layoffs #employment #jobs https://lnkd.in/eXgdBj-W

20

1 Comment

The world of AI models continues to move at pace - and #Foundry remains the place to go for the widest choice of models. Grab both performance and cost efficiency with Claude Sonnet 4.6.

9

Solid (AI for Data) is out of stealth! I am excited to share the news about our portfolio company's public launch with $20M in seed funding! ✨ Solid is tackling the biggest bottleneck in Enterprise AI: the lack of context. They are building the AI enablement layer, that moves enterprises’ AI from pilot to real-world production. Huge congratulations to ProFounders Yoni Leitersdorf and Tal Segalov and the entire Solid team. I enjoyed being part of your early ideation and company-building journey! Now let's scale it big time 🚀 This is a generational company at the foundation of the enterprise AI stack, reflecting our belief that the AI era is defined by reliability and results. #SuccessbyDesign Read more: https://lnkd.in/dX6JH6J5 Aviad Harell, Amir Zilberstein, Nathaniel Tavisal, Omri Sela, Noa Bar-Yosef, Robert Wiseman, Nick Aharoni, Noa Hen, Tomer Tirosh, Alon Melcer, Jonathan Bergerbest, Omer Biran, Tal Levi, Aviv Turecki, Liran Grinberg, Ori Barzilay, Ori Yankelev, Asaf Azulay, Tal Levi, Aviv Turecki

89

3 Comments

Less than 6 months after joining the Coralogix family, the Aporia team is back with a bang: Olly. Olly is an AI-native Observability Agent that turns observability data into actionable insights. * 3 months → integrated Aporia into the Coralogix platform (aka AI Center). * <6 months → a brand-new offering that redefines Observability. We’re just getting started. Watch the demo, share your thoughts, and feel free to guess what's next on our roadmap. 😉

363

23 Comments

𝗛𝗼𝘄 𝗠𝗖𝗣 𝗦𝗲𝗿𝘃𝗲𝗿𝘀 𝗠𝗮𝗻𝗮𝗴𝗲 𝗦𝗲𝘀𝘀𝗶𝗼𝗻𝘀 𝘄𝗶𝘁𝗵 𝗟𝗟𝗠𝘀 𝗮𝗻𝗱 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗪𝗶𝗻𝗱𝗼𝘄𝘀 Managing sessions in LLMs isn’t just about keeping track of prompts. It is about orchestrating memory, context, and tools seamlessly across multiple agents. That’s where MCP servers come in. Here’s the flow I captured visually → Context Window as Working Memory: Holds system prompt, user inputs, assistant responses, tool calls, and rolling context → MCP Servers: Each server maintains its own session state, ensuring consistency across interactions → System Prompt + Tools + Code Loops: Work together to power agentic behavior while grounding outputs in real tasks → LLM Integration: The context is constantly summarized and updated before being sent to the model This structure ensures scalable, multi-session orchestration, which is critical for enterprise AI where multiple tasks and tools interact simultaneously.

148

14 Comments

Parallelism with asynchronous off-policy learning will be the key to post-training with RL to expand beyond code and math reasoning.

9

The most interesting portion of the paper to me is chapter 4.2, a significant optimization for inference workload on a small GPU cluster affordable to midsize enterprise on-prem deployment. This design ensures a balanced computational load across devices, substantially boosting overall throughput, particularly during the inference phase. A key enabler for this advancement is Huawei's unique full-stack ownership of the NPU hardware (specifically the Ascend NPU platform), kernel, operating system and the pangu LLM. This vertical integration empowers Huawei to develop highly specialized optimizations, including the creation of innovative fused operators like MulAttention and SwiftGMM, which are custom-engineered to dramatically accelerate model inference specifically on the Ascend NPU platform. https://lnkd.in/gKiNYpat

7

1 Comment

CtrlB – changing orchestration of telemetry data.   Enterprises generate terabytes of logs every day, but traditional data architectures make it impossible to store and search this data comprehensively. CtrlB’s disruptive architecture is questioning these long-held industry assumptions and have built a platform that delivers 10x faster search at 90% lower cost.   Excited to announce our investment in CtrlB’s $2.5M seed round and thrilled to join Adarsh Srivastava & team CtrlB in their journey!   More details of why we invested in CtrlB - https://lnkd.in/gZNEVzER (by Naimil Shah)   Aman Raj Chiratae Ventures

154

4 Comments

A More Principled Metric for RAG Evaluation Evaluating retrieval-augmented generation systems has long been entangled with downstream performance metrics. Existing approaches often assess the quality of the final output, which makes it difficult to isolate the contribution of retrieval. Others rely on retrieval-specific metrics like NDCG or recall\@k, which are agnostic to how the retrieved context is actually used by the language model. SePer (Semantic Perplexity Reduction) proposes a more direct and model-aware alternative. It quantifies how much the retrieved documents reduce the semantic perplexity of the model on the input prompt. In effect, it asks the model: given this prompt, do these documents reduce your uncertainty? This is done without relying on gold answers or generated completions. The model computes a semantic probability distribution over possible answers before and after conditioning on the retrieved documents. The reduction in perplexity is treated as a measure of how useful the context was for informing the model’s internal representation of the query. Empirical results show that SePer correlates more strongly with human preferences than standard retrieval metrics. On five open-domain QA datasets, SePer achieved a Kendall tau correlation of 0.43 with human rankings. By comparison, NDCG\@20 reached only 0.20. Other common metrics, such as recall\@k, precision\@k, and reranking scores, performed similarly poorly. Notably, SePer maintains high correlation with human judgment even when varying only the retriever while holding the generator fixed. This suggests it can serve as a standalone retriever evaluation metric, decoupled from full-pipeline performance. Because SePer is generator-aware but does not require final output generation, it enables more targeted, efficient, and interpretable evaluation of retrieval in RAG systems. Rather than inferring retrieval quality through surrogate signals, it leverages the model’s own internal distribution as a source of supervision. Find the paper here arxiv.org/abs/2503.01478 #RetrievalAugmentedGeneration #LLMEvaluation #RAGMetrics

1

This is one of those things that just makes sense. The Linux Foundation (and by extension the PyTorch Foundation too) is a great home to nurture open-source projects.

4

Nvidia GTC presentation about structured & unstructured data being foundation for AI/LLMs used for business needs. We at OLake strongly believe the same and developing features around Iceberg. Apache Iceberg as per Feb 2026 release will soon be supporting unstructured data along side with structured and semi. https://lnkd.in/geYYF--6

22

Session date: May 01, 2026, noon CST Math challenge: Can you explain Compton’s ring experiment? New prompting with Andrew Ng (repo) How to prepare your data for the future life with Snowflake or FreeEed (repo) AI News Beyond Vibe: more spec-driven development (repo) How to build with Anthropic Claude Console (course)

2

1 Comment

Join the Databricks Data and AI Experts: Steve Sobel --Global Leader, Startup & Venture Programs; Kevin Rasmussen -- Field CTO, Early Stage Programs, and Sriharsha Tikkireddy -- Technical Lead, Early Stage Programs, for the next Omaxn event: Workshop: #VibeCoding Unlocked: Effortless App Building with Databricks Details" TechNexus Venture Collaborative on Aug 26 What you'll learn: #FasterDevelopment and #Iteration: Databrick's "#Vibecoding" allows developers to use natural language prompts to generate code and build applications, cutting the time from idea to deployment from weeks to minutes or hours. Engineers can test, refine, and redeploy applications in a highly efficient, conversational way—enabling tight feedback loops. #FullCustomization and #CodeOwnership: Engineers get complete control over the generated code instead of being limited by drag-and-drop low-code templates. You can modify, extend, and host the application anywhere, ensuring flexibility for enterprise needs and long-term maintainability. #PatternExtraction and #Maintainable Architecture: The AI focuses on extracting repetitive code patterns and parameterizing them, producing clean, documented configurations. This ensures that applications are not only quick to build, but also easy for future engineers to understand and maintain. #Integration with #DatabricksInfrastructure: Applications inherit benefits from Databricks’ robust governance and security: access controls, audit trails, RBAC, and deployment on the trusted Lakehouse platform. #TurningIdeas into #Apps: Fast: Vibe coding enables business stakeholders, analysts, and subject matter experts to describe their needs in plain English (or any natural language). The system translates these ideas into working applications, so non-techies can rapidly bring their insights to life—no computer science degree required.

10

1 Comment

AWS Trainium receives insufficient attention. While most of the industry has been laser-focused on NVIDIA GPU availability, Trainium has been quietly offering a compelling cost and performance profile — with very little of the open-source tooling needed to use it effectively for LLM inference. Our team just took a real step toward fixing that with Mini-SGLang-Neuron — a lightweight framework that brings SGLang to AWS Trainium and Inferentia silicons, with features like radix attention caching, chunked prefill, and tensor parallelism across Neuron cores. Benchmarked against vLLM-Neuron and available to run on Trainium1 instances through the Yotta console today. The broader point: the teams building hardware-agnostic inference infrastructure right now are positioning themselves well for a world where compute is abundant but fragmented. Trainium is one piece of that puzzle worth paying attention to. Open source — repo linked in comments.

2

1 Comment

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,672

190 Comments

"Taken together, these aren’t just high-profile database deals but proof that the AI infrastructure race is moving down-stack. While sophisticated models may still grab headlines, the battle is now increasingly about who can serve AI-ready data, fast, resiliently and at scale." An article from Forbes on how 3 recent database acquisitions are starting to prove a prediction that we made in March: Data is the undervalued element of AI. 👉 https://lnkd.in/dSWMak-J

12

2 Comments

🚀 Try out one of the fastest forecasting models in the market (15–50× faster than SOTA) — TTM-R3 and TTM-R3-Lite, with state-of-the-art accuracy 📈⚡

16