Swaroop Krothapalli

Ever feel like today’s AI models are getting smarter but somehow less useful? You’re not imagining it. LLM benchmarks have become the obsession. Model makers are locked in a cycle of chasing leaderboard points on MMLU, GSM8K, and other tests no one actually cares about outside of research labs. The result? Models that ace exams but fall apart in real-world use. They misinterpret vague instructions. They forget what you said three messages ago. They can’t think beyond a single step. But sure, the score went up by 0.3 percent. Benchmarks were supposed to be a guide. Instead, they’ve become a crutch. A way to show “progress” without improving anything that matters. The questions we should be asking are simple: Can the model follow messy, human instructions? Can it reason through problems, not just regurgitate answers? Can it actually help people get real work done? If you’ve worked with these models, you’ve felt the gap. You’ve seen the hype, and then you’ve seen it fall short when it counts. Are we building for people, or are we building for points? I’d love to hear what others are seeing in the wild. #AI #LLMs #Benchmarks #MachineLearning #RealWorldAI

27

3 Comments

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

As someone deeply invested in agentic systems and reinforcement learning, I find the Chain-of-Agents (CoA) approach both refreshing and practical, it takes the chaos of multi-agent orchestration and distills it into something elegant: a single model with native agent-like behavior. Here’s what I see: Traditional multi-agent setups have been great for modularity and reasoning, but they come with a high tax, too much inter-agent communication, redundant memory syncing, and unnecessary tool calls. What CoA proposes is radical in its simplicity: distill the behaviors of successful multi-agent runs into one model, then train that model to act as a team, internally. They start with multi-agent distillation: successful traces from existing orchestrators like OAgents are transformed into CoA-style traces, complete with planning steps, tool usage, reflections, and execution paths. But they don’t stop at replay, they filter hard cases and focus on high-quality trajectories that demonstrate useful agent behaviors like tool efficiency and coherent planning. This becomes the foundation of their SFT phase. Once that base is established, Agentic RL kicks in. The model is tasked with solving complex reasoning problems and importantly, ones where tools make or break the outcome (web QA, coding, math). Rewarding is done via exact-match or LLM-as-Judge strategies, making it possible to scale reward signals across noisy tasks. This setup tunes the model to reason reflectively, use tools judiciously, and respond with stability under diverse challenges. The kicker: by collapsing the roles of multiple agents into a single coherent model, CoA delivers an 84.6% reduction in inference cost. It’s a major unlock for production-grade agent systems where latency, memory, and tool limits matter. What excites me most is how CoA generalizes the spirit of ReAct/TIR, but upgrades it, dynamically invoking roles, using tools only when needed, and preserving a unified memory state that avoids the “who’s speaking now?” confusion. To me, Chain-of-Agents is a promising new frontier: a single-model, multi-mind architecture that captures the power of coordination while optimizing for real-world deployment constraints. And that’s where agentic AI needs to go. https://lnkd.in/exTxvsDD

63

2 Comments

One of the most transformative innovations powering today’s GenAI revolution is the advancement of foundation models (FMs) — large models pre-trained on Internet-scale data that demonstrate remarkable generalization across a wide range of tasks. FMs have redefined performance benchmarks in many fields including, natural language processing, computer vision, and computational biology, becoming the cornerstone of modern GenAI. Yet, progress on structured enterprise data such as tabular time series has lagged behind, where task-specific models optimized on manually extracted signals still dominate. Inspired by the success of FMs in other domains, we set out to explore whether FMs can also be built for finance and banking. After over three years of R&D, I’m excited to share that we’ve deployed our first structured-data FM into production at TD: TD AI Prism https://lnkd.in/gEAb7Scm. Trained on billions of customer events, Prism generalizes to many downstream predictive tasks, delivering higher accuracy and significantly faster time to market. Through this work, we're beginning to observe early signs of scaling laws, similar to those seen in LLMs, where more data and larger models lead to stronger generalization. As part of this effort, we also released a public FM trained on Internet tabular data, capable of making predictions on new datasets without any fine-tuning. Check it out here: https://lnkd.in/gmbWdAf7 As enterprises evolve with AI, we believe that foundation models for structured data will be a major force in redefining how predictive AI is deployed at scale, driving the next wave of innovation in enterprise applications. Layer 6 AI has multiple exciting initiatives in this space, so stay tuned for more updates!

360

15 Comments

Proud to share my latest blog post on linear regression! 📈 Dive into the fundamentals of this key statistical method and explore how it models variable relationships. Perfect for data enthusiasts! Read more: https://lnkd.in/gm-aGnxz

18

1 Comment

A Capital One AI research team won Outstanding Paper at the 2025 AAAI conference! The paper introduces methods for protecting LLM-powered conversational agents from jailbreaks and other malicious inputs. Check out the techniques on our tech blog. https://bit.ly/45GOTEM

81

3 Comments

In the context of Path over Graph (PoG) for LLM-based multi-hop reasoning, various graph traversal techniques are employed to discover meaningful reasoning chains across a knowledge graph. Breadth-First Search (BFS) explores all immediate neighbors of an entity before moving deeper, ensuring that short but relevant paths are not missed—ideal for shallow reasoning tasks. Depth-First Search (DFS) goes as far along each branch as possible before backtracking, which can uncover deeper, more complex chains of reasoning. Beam Search strikes a balance by maintaining only the top-k most promising paths at each hop based on a scoring function (such as semantic similarity to the question), thereby controlling combinatorial explosion. After traversal, Path Ranking evaluates the semantic relevance of each candidate path—typically by converting them into natural language statements and scoring them against the query using models like SBERT or LLMs.. Beam Search is a controlled BFS that balances exploration and efficiency. At each hop, it keeps only the top-k highest scoring paths based on a scoring function f(pi,q)f(p_i, q)f(pi​,q), where pip_ipi​ is a candidate path and qqq is the question. The score could be computed using cosine similarity between SBERT embeddings: This scoring prunes irrelevant branches and helps find semantically relevant reasoning chains efficiently. Path Ranking - Once a set of candidate paths is retrieved, each path pip_ipi​ is converted into a natural language sentence (e.g., using templates), and its semantic similarity to the question qqq is computed. The paths are then ranked using: Technique Description Breadth-First Search (BFS)Explores all neighboring nodes before going deeper — good for finding short, relevant paths. Depth-First Search (DFS)Goes deep into the graph first — might find longer reasoning chains. Beam SearchKeeps the top-k most promising paths at each hop based on a scoring heuristic (e.g., similarity to query). Path RankingAfter traversal, paths are ranked based on semantic closeness to the original question. Why Is It Important? LLMs do not inherently know how entities are related in a structured KG. Traversal over the KG's structure allows PoG to find explicit, factual reasoning chains. These chains can later be passed into the LLM to ground the answer in factual paths. #POG #LLM #GENAI #GraphTraversal #LLMReasoning_Graph

25

🚀 Just wrapped up a hands-on project applying Retrieval-Augmented Generation (RAG) to a real-world use case: interpreting LIC’s New Jeevan Shanti policy document using LLMs. The core idea? Combine vector search over regulatory PDFs with GPT-4 to generate grounded, faithful answers to complex policy questions. Key capabilities we built: ✅ LCEL-based RAG architecture using LangChain ✅ Contextual compression retriever for focused document lookup ✅ Modular LLM & embedding selection (OpenAI, HuggingFace, Mistral) ✅ Evaluation using LLM-powered metrics: faithfulness, relevance, correctness 📄 The document used is publicly available here: LIC New Jeevan Shanti Policy PDF 💻 Full code available on GitHub: https://lnkd.in/gS9mEzWA 💡 This architecture brings the best of both worlds — retrieval accuracy and generative flexibility — especially in regulated domains like insurance and financial services. 🔜 Next step: Incorporate Agents that can reason across multiple steps — not just answer a question, but decide which tools to use, when to search, how to validate, and when to stop. That’s where the future of enterprise GenAI lies. Would love to hear how others are pushing RAG, agents, and LLMs in production! #GenAI #LangChain #LLM #RAG #AIForOps #InsuranceTech #LLMEvaluation #LangChainAgents

61

1 Comment

Curious About Regression Discontinuity Design (RDD)? I just released a new chapter on Regression Discontinuity Design (RDD) -- an elegant tool in causal inference. Consider a scenario where a school district enforces a policy that students with grades below a cutoff are enrolled in a tutoring program, while those above are not. After some time, the question arises: how can we evaluate the program’s effectiveness? We probably can compare the performance of students near the cutoff — those just above and those just below — since they are similar in all respects except program participation. This approach is known as Regression Discontinuity Design (RDD). In the tutoring program example, the cutoff is a hard threshold — but what if the cutoff is soft? For instance, students with grades below the cutoff might still have a higher probability of enrolling in the tutoring program, even though participation is not strictly enforced. The strict enrollment case is known as "Sharp RDD", while the probabilistic enrollment case is called "Fuzzy RDD." These two designs differ significantly in their econometric approach: Sharp RDD estimates the performance difference at the cutoff directly, whereas Fuzzy RDD requires the use of Instrumental Variables (IVs) to account for imperfect compliance. That’s why this book introduces RDD after Chapter 7: “Instrumental Variables”. Since soft cutoffs are even more common in real-world applications, it is essential to understand how to implement Fuzzy RDD in detail. Let me know your thoughts! https://lnkd.in/emnxZKmS

13

🎉 📄 Delighted to share our new article titled "Dynamic model selection in enterprise forecasting systems using sequence modeling," co-authored with Jinhang Jiang and Karthik Srinivasan, published in Decision Support Systems (DSS)! In this work, we studied how to identify the best model(s) from legacy forecasting systems for individual time series over the long term—a key challenge in large-scale forecasting. We proposed a dynamic model selection framework that adapts a NLP-style sequence modeling to learn and generate switching patterns directly from historical model performance, offering a proactive, scalable, and adaptive solution for enterprise environments. #AI #forecasting #dynamicmodelselection #timeseries #decisionsupportsystems https://lnkd.in/gKqucj_p

39

1 Comment

Reinforcement Learning (RL) is about algorithms that learns to achieve a goal through trial and error, adapting its behavior based on its experience of interacting with its environment. However, LLMs largely discarded the methodology of experiential RL in favor of training on massive amounts of human-generated data. While techniques like RLHF became popular for aligning LLMs, these approaches often bypass core RL concepts such as value functions (relying on human experts instead of machine-estimated values) and exploration (depending on strong priors from human data rather than discovering new behaviors). This reliance on existing human knowledge imposes a ceiling on the agent's performance, preventing it from going beyond what humans already know and limiting its ability to achieve superhuman intelligence. To overcome this limitation one has to return to classic RL concepts, adapting them for agents that learn continuously from long streams of grounded, autonomous interaction with the real world and its diverse signals. This approach aims to enable agents to autonomously discover knowledge and strategies that go beyond human limitations. "Welcome to the Era of Experience" - David Silver, Richard S. Sutton

11

2 Comments

In collaborative machine learning (CML), early contributors take on more risk and encourage participation from wait-and-see parties, but existing CML frameworks treat every party as if they join simultaneously. The #NeurIPS2025 paper of Jiangwei Chen, Nguyen Pham, Rachael Sim, Arun Verma, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low proposes a time-aware CML framework that rewards parties not just for what they contribute, but when they contribute: •⁠ ⁠8 incentives extending fairness with time-aware notions that incentivize early participation while preventing low-quality early submissions. •⁠ ⁠2 reward distribution methods that theoretically satisfy all incentives and recover the #ShapleyValue in time-agnostic cases. •⁠ ⁠A practical recipe for constructing valuation functions and reward realization, with empirical validation on synthetic and real-world data. Find our more at our posters! #NeurIPS2025 Thu, Dec 4, Exhibit Hall C, D, E #1104 #EurIPS2025 Wed, Dec 3, Poster Stand #56 Paper: https://lnkd.in/g_yM6j39 #FederatedLearning

50

Brilliant post by Jessica Talisman on the difference between semantic layers and ontologies. The frame couldn't be clearer: - Semantic layers solved for metric governance in a world where users interacted with dashboards - Ontologies provide a structured representation of knowledge where AI systems can access context and make inferences Full article here: https://lnkd.in/eBDyMUfa

14

If you’re an AI engineer or an AI leader looking to level up your skills in LangChain and LangGraph, this one is for you. I’m teaming up with Arvind Narayanamurthy for a beginner–intermediate, 1.5 hour hands-on workshop where you’ll learn how to move from simple prompts to real AI applications and agentic workflows. Why this matters - Most engineers know how to prompt. Few know how to orchestrate. - LangChain and LangGraph are becoming the backbone of production AI systems. - If you want to build or lead AI engineering teams, you need these mental models. What you’ll learn - Foundations of LLM orchestration and where LangChain fits - Essential building blocks: chat models, prompting patterns, structured outputs, LCEL - Tool calling with MCP adapters - RAG basics end to end - LangGraph workflows and agentic architectures - Two live labs: a micro-app and a simple agent - Quick dive into LangSmith for debugging The workshop is recorded, and everyone gets the replay. If you want practical, real-world skills to build modern AI systems, this is the one to sign up for. Grab your seats: https://lnkd.in/eW2WZdcf

328

36 Comments