Rami Krispin

Retrieval-Augmented Generation (RAG) is often described as “connecting an LLM to external knowledge,” but this framing hides what actually matters architecturally: how retrieved text influences the model’s final response during generation. Retrieval-Augmented Generation (RAG) works not by letting an LLM “look things up,” but by conditioning token generation on retrieved evidence. A user query is embedded, relevant chunks are retrieved from a vector store, and those chunks are injected into the prompt as plain text. From the model’s perspective, this is a single sequence of tokens—system instructions, retrieved chunks, and the user question—processed together. During generation, the LLM predicts each next token using self-attention over the entire context. Retrieved chunks matter because they reshape the probability distribution of possible next tokens, increasing the likelihood of fact-grounded outputs and reducing hallucinations when the signal is strong. Formally, P(answer | query, retrieved_chunks) ≠ P(answer | query), highlighting that the core architectural insight of RAG is that it updates context, not model weights, making chunk quality and relevance critical to the final response. If this helped clarify RAG beyond the buzzwords, feel free to reshare. #RAG #LLM #AIArchitecture #GenAI #MachineLearning

18

🚀 When More Isn’t Better in AI Vector databases are the quiet workhorses of modern AI — they store the embeddings that make semantic search, RAG, and personalization possible. But there’s a hidden trap: 📉 In high-dimensional spaces, everything starts to look the same. 📈 Your compute bill climbs while accuracy drops. This is the curse of dimensionality — and it’s why your 1,536-D embeddings might be slowing you down more than you think. In my latest post, I unpack: Why distances “collapse” in high-D space How it breaks ANN search 4 practical ways to fight back A real-world benchmark that cut index size by 75% with almost no recall loss 💡 If your AI stack depends on retrieval, you’ll want to see this. 👉 Read the full article: https://lnkd.in/euMa767T #ArtificialIntelligence #MachineLearning #AI #VectorDatabases #Embeddings #RetrievalAugmentedGeneration #SemanticSearch #ANNsearch #DataScience #BigData #DataEngineering #DatabaseOptimization #InformationRetrieval #AITech #TechInnovation #ScalingAI #PerformanceOptimization #Pinecone #Milvus #Weaviate #FAISS

1

The person who built Apache Spark just won the ACM Prize in Computing. Then he said AGI is already here. Matei Zaharia, Databricks co-founder and CTO, received the 2026 ACM Prize, the second-highest honor in computer science. The recognition is for creating Spark, the distributed computing framework that runs inside most enterprise data pipelines today. What Zaharia said after winning is more interesting than the award itself. He told TechCrunch that AGI is here already, but not in a form that we appreciate. His argument challenges how we measure intelligence. We keep measuring AI against human-centric benchmarks and miss the fact that AI systems already exceed human performance on specific, bounded tasks. We are looking for a general intelligence that looks like us. ↳ Instead of recognizing the specialized intelligence that already outperforms us. This matters because Zaharia runs the technical direction of a $134B company doing $5.4B in revenue. With a likely H2 2026 IPO that could be the largest enterprise software listing in history. His view of AGI shapes what Databricks builds next. Including Genie Agent Mode, a multi-step analytical agent that shipped this week alongside a mobile app. The arc from Spark to AGI-debate to a $134B valuation tells you something about where enterprise data infrastructure is heading. ➟ The data warehouse was step one. ➟ The AI agent sitting on top of it is step two. What is the most useful task an AI agent does better than a human in your data workflow today. And would you call that AGI or just good tooling? - - - - - - ♻️ Found this useful? Repost ➕ Follow me for more

3

2 Comments

"GenAI interpretability gets less attention than the constant deluge of model releases, but it is arguably more important." I completely agree with this point from the recent Urgency of Interpretability blog by Dario Amodei. As foundation models evolve rapidly, the lack of interpretability can become a major barrier to safe, compliant, and large-scale adoption—especially in finance. To build trust, we must be able to explain model outputs clearly and reliably. I’m excited to speak on this topic at the upcoming AI4 2025 conference, where I’ll present our latest work: “Beyond the Black Box: LLM Interpretability in Finance.” I’ll be sharing how we’re getting into the "brain" of LLMs to uncover meaningful financial features and apply them across key use cases like trading and sentiment analysis. Thankful to my co-author, Ariye Shater, for his collaboration on this work. Links: - Paper Link: https://lnkd.in/exrmXh97 - Conference: https://ai4.io/finance/ - Dario's Blog: https://lnkd.in/ePDp-x-W #GenAI #Interpretability #Finance #LLMs #AI4Conference #SparseAutoencoders #Anthropic #ResponsibleAI #MechanisticInterpretability

87

How do you know your LLMs are actually ready for enterprise use? 🤔 LLM and agent evaluation is becoming one of the hardest, and most important, challenges in production-grade GenAI systems. Most teams still rely on static benchmarks that quickly go stale or fail to capture real-world workflows, or use human evaluations which are not scalable. Enterprises need evaluation frameworks that reflect their own data, context, and decision loops. That’s why I found this new paper from Snorkel AI interesting: 📄 Automating Benchmark Design (BeTaL) — https://lnkd.in/gBJ2UD4c BeTaL proposes a method to automatically tune benchmarks using LLMs-in-the-loop, optimizing for difficulty, coverage, and realism. ✅ Reduces deviation from target difficulty ✅ Delivers 2–4× better consistency than baseline methods For enterprise GenAI systems — especially in regulated domains — custom, dynamic benchmarks may soon be as essential as model fine-tuning. What’s your biggest pain point when evaluating LLMs or agents in your organization? #LLM #Benchmarking #GenAI #EnterpriseAI #AIResearch #Evaluation

2

4 Comments

Monday we open-sourced AI Functions in Strands Labs, today we shared a blog that hints at how they are key to making AI Agents operate as Universal Task Solvers. Tomorrow we will explain these concepts in the CS Colloquium at Princeton, and Friday at Columbia. Join if in the area. https://lnkd.in/gqyu2b5w https://lnkd.in/gvP8pAre

136

5 Comments

Most Large Language Models stop learning the moment training ends. They predict well—but they don’t adapt. In my latest blog, I explore Dragon Hatchling (BDH), a brain-inspired architecture that rethinks how AI learns, remembers, and reasons. BDH introduces inference-time learning, synaptic memory, interpretability by design, and stable long-horizon reasoning—pushing LLMs beyond static text generation toward truly adaptive systems. If you’re interested in the future of: • Long-running AI agents • Lifelong learning systems • Interpretable & modular AI • What comes after Transformers 👉 This is worth a read. 🔗 https://lnkd.in/gSs4VJYs #AI #LLMs #ArtificialIntelligence #MachineLearning #FutureOfAI #BDH #AgenticAI #DeepLearning

4

From the front line of delivering Agentic systems Forget the pilot and the launch party. The real game of agentic AI starts when it goes live. In 2026, a lot of Agentic systems are shifting from isolated pilot programs to full enterprise-scale deployment. Being in the middle of this shift, I see these areas to focus on 1. Token Discipline. This is the key to control cost for Agentic systems. Treating tokens like a precious currency and implementing real-time metrics for every agent request. Without this, costs spiral and the ROI vanishes. Tokens are the measure of Cost Of Goods Sold (for those of you in finance LOL). 2. Think agent Observatory, not dashboards. Develop or integrate with systems that evaluate agent accuracy, detect subtle failures, and trigger automated playbooks is top of mind. I see observability as the smoke detector for customer trust. Without one we are flying blind, and the first one who will notice that our system doesn’t deliver value is…our customer. 3. Institutionalize Lineage. This often overlooked. Every data source, prompt version, model iteration, reasoning result and evidence should be traceable. Version everything. And in all this, my observation is also that the definition of the secret sauce, our intellectual property, and competitive advantage, has shifted from traditional AI models a few years ago to the agentic reasoning prompts and the data layers and knowledge graphs powering these agents. Interesting times! #AIStrategy #AgenticAI #LiveAgents 2026

22

Deep Research is emerging as a common paradigm across applications for both consumer search use cases and enterprise knowledge work. This work explores the application of Deep Research techniques to recommendation systems. https://lnkd.in/g3Y2vtKD

40

Introducing "Building with Llama 4." This short course is created with Meta, and taught by Amit Sangani, Director of Partner Engineering for Meta’s AI team. Meta’s new Llama 4 has added three new models and introduced a Mixture-of-Experts (MoE) architecture to its family of open-weight models, making them more efficient to serve. In this course, you’ll work with two of the three new models introduced in Llama 4. First is Maverick, a 400B parameter model, with 128 experts and 17B active parameters. Second is Scout, a 109B parameter model with 16 experts and 17B active parameters. Maverick and Scout support long context windows of up to a million tokens and 10M tokens, respectively. The latter is enough to support directly inputting even fairly large GitHub repos for analysis! In hands-on lessons, you’ll build apps using Llama 4’s new multimodal capabilities including reasoning across multiple images and image grounding, in which you can identify elements in images. You’ll also use the official Llama API, work with Llama 4’s long-context abilities, and learn about Llama’s newest open-source tools: its prompt optimization tool that automatically improves system prompts and synthetic data kit that generates high-quality datasets for fine-tuning. If you need an open model, Llama is a great option, and the Llama 4 family is an important part of any GenAI developer's toolkit. Through this course, you’ll learn to call Llama 4 via API, use its optimization tools, and build features that span text, images, and large context. Please sign up here: https://lnkd.in/gXKeipht

2,830

111 Comments

What does autonomous AI look like in the physical world, where every correct AI output generates profit, but every hallucination hurts the bottom-line? Databricks' Shiv Trisal chatted with me about this and many other topics around data and AI in supply chains, energy, logistics, and more. Enjoy the short clip, as well as check out the full 43-minute interview: https://lnkd.in/gnSm-wF3 Thank you again Shiv. Also thank you Philip Tam for helping me get this room, the best setup I've gotten the chance to record in!

62

6 Comments

The Most Important AI Chart You Haven’t Paid Attention To METR just published a breakthrough analysis on how long of a task today’s frontier AI models can actually complete. Most AI benchmarks measure accuracy, latency, or reasoning. But almost nobody measures task horizon — how long of a multi-step task an AI model can reliably finish. METR’s new study does exactly that. And the trend is staggering. ➡️ AI’s “Task Horizon” Is Doubling Every ~7 Months The graph shows how long a task (measured in human-hours) today’s models can complete with a 50% success rate. Models crush tasks that take a human <4 minutes. Success drops sharply as tasks approach hours. But the horizon is expanding fast — doubling roughly every 7 months for the last several years. If this trend even partially holds, we’re on track for agents that can autonomously execute hours-long workflows within 12–18 months… and days-long workflows not long after. This is the real inflection point people aren’t talking about. ➡️ Why This Matters Benchmarks like MMLU, GSM8K, or HumanEval tell us only about point-in-time reasoning. But real work — marketing ops, merchandising, design workflows, engineering tasks, GTM processes — are multi-step, interrupt-driven, and span hours or days. Today’s LLMs are brilliant sprinters. But this study shows they’re becoming marathon runners — fast. ➡️ But There’s a Big Catch Production systems need 80–90% reliability, not 50%. At that bar, the task horizon is still much shorter. Which means: The winners in this next phase aren’t the ones with the biggest model — but the ones with the best orchestration, tool-use, supervision, and agent design. ➡️ What This Means for Agentic AI Builders If you’re building agentic systems, take note: # Design for layered horizons — minutes today, hours tomorrow. # Decompose long workflows into fail-safe steps. # Instrument everything — logs, breakpoints, tool traces. # Prepare for an era where models manage end-to-end workflows, not just answer prompts. # Your differentiator won’t be the model. It will be orchestration, reliability, and guardrails. ➡️ The Quiet Signal Behind the Noise While the industry chases flashy releases, this METR analysis might be the most important signal of where AI capability is truly heading — and how fast. AI task length capability is compounding. And compounding curves always surprise us… only in hindsight. 👉 link: https://lnkd.in/eWP-8qXk

11

A few years back, I sat through a meeting brimming with buzzwords—“deep neural networks,” “reinforcement learning,” & “transfer learning,. Everyone nodded along, but many left more confused than enlightened. That’s #JargonAI in action: using terms that sound smart but often say very little. It’s not just a minor issue. A study from Ohio State University found that too much jargon actually makes people understand less and trust less (https://lnkd.in/gvtFRRRi?) This hit me hard. Since then, whenever I deliver anything—whether it’s a presentation, a report, or just a quick update—I make sure everyone in the room can understand it. If someone outside the tech team can’t follow, I simplify it again. Because communication isn’t about showing how much you know. It’s about helping others know more. Let’s make #clarity a #leadershiphabit. #AILeadership #Communication #AI #JargonAI

69

1 Comment

WIKIPEDIA also HIT BY AI Similarweb published stats of YoY traffic change and … it shows that Wikipedia (is similarly to Google Search, Stack Overflow and other “seek the answer” platforms ) seriously hit by LLM’s giving direct AI summary, rather than sending individuals to read the original info sources. I gonna attach a small hack. Do you want some topic to be represented in LLM training sets. Well, then create a wikipedia page about the topic and you are “guaranteed” to swim into training data.

74

9 Comments

Are AI agents the solution or just another layer of complexity: • As enterprise adoption grows, the real question is trust • Research shows AI agents are transforming enterprise workflows but require clear boundaries per Box CEO Aaron Levie, who warns context overload can derail productivity • Google's Gemini CLI integration with Zed highlights how AI is becoming a co-pilot for developers, enhancing collaboration and speed • The rise of tools like Endex for Excel shows AI is democratizing advanced capabilities for everyday business tasks, per OpenAI’s latest funding #AIInTheWorkplace #FutureOfWork #EnterpriseAI #DigitalTransformation #AI

3

TL;DR: A/B tests validate. Counterfactuals scale. How are you thinking about this? As organizations move from traditional analytics to Agentic AI systems, one question keeps coming up: Should we rely on A/B testing or counterfactual inference to understand impact and guide optimization? Both matter - but they serve very different purposes. Evaluating impact isn’t just “did it work?” It’s: “what would have happened if the agent chose differently?” 🔹 A/B Testing Classic causal validation: controlled, randomized, clean comparisons. Works well for single, isolated decisions - but agentic systems operate in dynamic, multi-step environments with branching trajectories. A/B tests alone can’t capture this complexity. 🔹 Counterfactual Inference Estimates “what would have happened under alternative decisions” using tools like: • Causal forests • Uplift modeling • Synthetic controls • Off-policy evaluation These methods allow scalable, multi-step causal reasoning without needing dozens of live experiments. 🛒 Retail Examples • Personalized promotions: Estimate if the agent’s recommended offer increased incremental sales vs. what would have happened with a different incentive. • Store operations agents: Measure how much faster labor scheduling or replenishment tasks would have been if the agent suggested an alternative pathway - without needing dozens of live experiments. • Member engagement journeys: Understand the lift of an agent altering sequence, timing, or tone of outreach across touchpoints. ✅ Takeaway A/B tests anchor big decisions. Counterfactual inference scales intelligence across agentic AI systems. For complex real-world environments, counterfactuals are essential for causal reasoning at scale. #DataScience #CausalInference #AgenticAI #RetailAnalytics #MachineLearning Swati Kirti Chittaranjan Tripathy Yog Seetharama Fabiano Battaglin Subhasish Misra Felipe Morales Heerlein Jason Cho, PhD

69

12 Comments