Context Laundering in Large Language Model Workflows

One of the biggest challenges I see with scaling LLM agents isn’t the model itself. It’s context. Agents break down not because they “can’t think” but because they lose track of what’s happened, what’s been decided, and why. Here’s the pattern I notice: 👉 For short tasks, things work fine. The agent remembers the conversation so far, does its subtasks, and pulls everything together reliably. 👉 But the moment the task gets longer, the context window fills up, and the agent starts forgetting key decisions. That’s when results become inconsistent, and trust breaks down. That’s where Context Engineering comes in. 🔑 Principle 1: Share Full Context, Not Just Results Reliability starts with transparency. If an agent only shares the final outputs of subtasks, the decision-making trail is lost. That makes it impossible to debug or reproduce. You need the full trace, not just the answer. 🔑 Principle 2: Every Action Is an Implicit Decision Every step in a workflow isn’t just “doing the work”, it’s making a decision. And if those decisions conflict because context was lost along the way, you end up with unreliable results. ✨ The Solution to this is "Engineer Smarter Context" It’s not about dumping more history into the next step. It’s about carrying forward the right pieces of context: → Summarize the messy details into something digestible. → Keep the key decisions and turning points visible. → Drop the noise that doesn’t matter. When you do this well, agents can finally handle longer, more complex workflows without falling apart. Reliability doesn’t come from bigger context windows. It comes from smarter context windows. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

Stop blaming the model. Fix the context. Most “broken” agents and RAG stacks aren’t model problems. They’re context problems. Wrong chunks. No memory hygiene. Too many tools in the prompt. Zero control over what actually reaches the model. In agentic systems, context is the job: → Agents decide what to load, what to drop, what to re-query. → RAG decides which 3–5 chunks get a golden ticket into the window. Blow that, and your “smart” agent is just an expensive guess machine. When we tightened context for a service workflow, hallucinations dropped, answers stopped drifting, ops finally trusted the system. The only “feature” we shipped was better context engineering and a clean write-back into sessions.context_state. Sharing this Weaviate context engineering guide because it nails the difference between “prompting a model” and “designing the world it thinks inside.” Save this if you’re serious about agents, not just chatbots. Link to the full report: https://lnkd.in/gXX4e2PM

Up until now, much of domain specific knowledge injection to LLMs has answered the question: "How do we get the right context INTO the window?", but with the success of coding agents and recursive language models, that question has changed to: "How do we let the model NAVIGATE context itself?" Large language models have a limited context window, or maximum amount of tokens that can be input as its entire context. This is a hard constraint resulting from the transformer architecture itself, and while modern models have pushed context windows into the hundreds of thousands (even millions) of tokens, more context doesn't always mean better results. Research has shown that model performance actually degrades as input length increases, a phenomenon known as context rot, where models struggle to reliably use information buried deep in long sequences, especially when surrounded by similar but irrelevant content. The solution up until now has been Retrieval Augmented Generation (RAG), chunking and embedding documents into vector databases, then retrieving the most relevant pieces via semantic similarity. This works, but it frames context management purely as a search problem, and scaling it starts to feel more like building a search engine than an AI system. What coding agents like Claude Code, Cursor, and Codex stumbled into was a different approach entirely: give the LLM a terminal and let it explore. Filesystem-based context navigation lets models directly explore, preview, and selectively load content using tools they already understand. Instead of engineering a pipeline to deliver the right context, the model finds it itself. Recursive Language Models (RLMs) formalize this further, with a slight distinction: in a coding agent, opening a file or running a tool dumps results back into the context window. RLMs instead store the prompt and all sub-call results as variables in a code environment, only interacting with them programmatically. Recursion happens during code execution, meaning the model can spawn arbitrarily many sub-LLM calls without polluting its own context, orchestrating understanding of 10M+ tokens without ever having to look at all of it at once. This gives us two differently motived options: RAG gives you fast, narrow retrieval great for latency-sensitive apps like chatbots. RLM-style frameworks trade speed for deeper exploration, better suited when thorough analysis matters more than response time. To learn more about context rot, how coding agents changed context delivery, and how recursive language models are formalizing it all, check out my latest video here: https://lnkd.in/ehszSKV7

Bigger context windows don't fix retrieval. They just move the problem. Everyone is chasing million token contexts and assuming more input equals better answers. But performance drops long before you hit the limit. Stanford’s "Lost in the Middle" study showed it years ago: LLMs handle information best at the start or end of their prompt, and tend to ignore what’s in the middle. A 2024 follow‑up found even worse degradation during multi‑hop reasoning. Models don’t just lose track of mid‑context details but struggle to connect related pieces. 𝗙𝗼𝘂𝗿 𝘄𝗮𝘆𝘀 𝘁𝗵𝗶𝘀 𝗯𝗿𝗲𝗮𝗸𝘀 𝘆𝗼𝘂𝗿 𝘀𝘆𝘀𝘁𝗲𝗺: 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗽𝗼𝗶𝘀𝗼𝗻𝗶𝗻𝗴 - Noisy or wrong information enters the window once and quietly gets reused later 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗱𝗶𝘀𝘁𝗿𝗮𝗰𝘁𝗶𝗼𝗻 - Long histories and stale tool output make the model lean on repetition instead of reasoning fresh 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗰𝗼𝗻𝗳𝘂𝘀𝗶𝗼𝗻 - Irrelevant documents crowd the window so the model latches on to the wrong chunk 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗰𝗹𝗮𝘀𝗵 - Contradictory or outdated facts sit side by side and the model "averages" them into something useless 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝘄𝗼𝗿𝗸𝘀: • Retrieve on demand instead of stuffing everything in  • Summarize history instead of storing it verbatim  • Prune aggressively because old context becomes a liability  • Put critical information at the start or end of prompts 𝗧𝗵𝗲 𝗴𝗼𝗼𝗱 𝗻𝗲𝘄𝘀: Model research is chipping away at positional bias, but it will never replace good retrieval and context hygiene. Context windows are working memory, not cold storage.

AI agents connecting to dozens of MCP servers face a critical bottleneck: every tool definition and intermediate result flows through the context window, burning hundreds of thousands of tokens for simple workflows. When you ask an agent to fetch a sales transcript from Google Drive and update Salesforce, the traditional approach loads all 50 tool definitions upfront at 150,000 tokens, then passes the entire 50,000 token document through the context twice. This creates a token consumption spiral that makes agents slower, more expensive, and increasingly error prone as you scale to more integrations. The result is context overflow, degraded accuracy, and workflows that simply fail when you exceed window limits. Anthropic published this research paper 1 month ago, I tried to look for some samples but did not find any, so built one that you can plug in and see in action! Code execution with MCP flips this architecture. Instead of loading all tools upfront, agents discover them progressively by exploring a virtual filesystem, reading only the specific tool definitions they need. Instead of passing large datasets through the context window, agents write Python scripts that execute in a secure sandbox where data stays local and only final results return to the model. Key features include: • Progressive tool discovery that loads definitions on demand from a ./servers/ directory structure • Sandboxed code execution environment where intermediate data never enters the context window. Here, we use Amazon Web Services (AWS)'s AgentCore Code Interpreter. • Type safe Python wrappers auto generated from MCP tool schemas for reliable agent code • Lazy loading that connects to MCP servers only when tools are actually called • Privacy by design since sensitive data processes locally without tokenization INSIGHTS 🚀 : I tested this on a task of analyzing all 83 repositories and identifying the top 5 most starred shows compelling results. The code execution approach used 28,776 tokens versus 114,078 for regular MCP, achieving 74.8% token reduction and 69.5% cost savings at $0.109 versus $0.357 per request. More importantly, code execution delivered exact star counts for all repositories with verifiable Python code generation, while the regular approach could only provide approximate ordering without actual counts due to API response limitations. The code execution method correctly identified all metadata including star counts of 6, 5, 3, 3, and 3 with primary languages and complete descriptions, generating executable code that fetches, sorts, and formats results for full transparency. Getting started with the Python implementation at: 1. Github repo: https://lnkd.in/eiJKbuYw (try it out and leave a star!) 2. Full technical deep dive: https://lnkd.in/e4dzU-SQ Chaitra Mathur Kosti Vasilakakis #aws #MCP #anthropic #agents #llms

We’ve spent the past year optimizing prompts and tuning models. But the real frontier isn’t what you ask an LLM. It’s what context you give it — and how you assemble that context at runtime. This goes far beyond RAG. We’re now designing systems that stitch together: • memory blocks • tool schemas • retrieved documents • system prompts • external API outputs all into the context window… in milliseconds. Think of it like a browser session: • Your prompt is the search bar • But your cookies, autofill, extensions, and open tabs all shape the final output Now imagine the browser has root access to your org. That’ll be the AI agent. What this likely means for security: • Context injection is the new prompt injection (e.g., poisoned docs, stale memory, manipulated tool metadata) • Retrieval poisoning can silently alter outcomes (e.g., embedding sensitive or false info in indexed sources) • Tool misrouting or schema abuse leads to incorrect, potentially destructive LLM actions • Agent memory drift can leak state across sessions or users if not scoped properly What we’ll likely need next: • Memory sanitizers and plugin firewalls • Context lineage tracing (what fed this output?) • Tool execution governance • Real-time MCP enforcement These are early thoughts — trying to learn as fast as the space is evolving. Would love to hear how others are thinking about securing the context window. Highly recommend: https://lnkd.in/g65SZQXG Letta for their latest blog https://lnkd.in/gYRmPAR2 LangChain for their latest blog https://lnkd.in/gNkDGFHX #AI #Security #ContextEngineering #PromptInjection #LLM #RAG #AIAgents #CopilotSecurity #MCP #AgentSecurity

Context Engineering: quietly becoming one of the most important design problems in AI-native development. We’re watching something interesting happen in real time: LLMs are getting smarter—faster reasoning, better planning, more fluid code generation. But they’re still not remembering well. So instead of waiting for “memory” to mature, we’re building systems around that gap. That’s where Context Engineering comes in. A common approach from teams is to start with AI-assisted coding or AI PR reviews, codify rules and guidance into monorepo or something highly accessible to the LLM, then achieve decent results with AI pair programming. Along the way prune dead code and docs and improve your system architecture where you can. That's essentially level 1 context engineering. ⸻ So what is Context Engineering? It’s shaping, curating, and delivering relevant information to a language model at inference time, so it can behave more intelligently—without actually learning or remembering anything long-term. We see this everywhere in dev tools now: • Cursor: thread-local + repo-aware context stacking • Claude Code: claude.md as a persistent summary of dev history • Prompt-engineered PRDs living next to source files • Custom eval + test suites piped into the session as scaffolding • Vector stores / RAG / MCP servers acting like external memory prosthetics I use all of these in my developer workflow day to day. It's a constant time and effort commitment to optimize for the best context for my AI coding assistants - currently using Claude Code primarily with Cursor as backup. Agents running in GitHub are Copilot with some autonomous troubleshooting with GPT-4.1. We're basically tricking the LLM, with a lot of effort, into feeling like it remembers, by embedding the right context at the right time—without overwhelming its token window. ⸻ Why this matters -> LLMs today are like brilliant interns with no long-term memory: You get great results if you prep them with your wisdom but constrain the thinking boundaries. -> Context Engineering becomes the new “prompt discipline”—but across system design, repo architecture, and real-time tooling. We’re not teaching models to remember (yet). This is a major AI gap-and something we're working on at momentiq. We’re teaching ourselves how to communicate, in a relatively inefficient manner, with high-leverage, stateless minds. Context Engineering is working for now and absolutely should be a focus for teams on the AI-native journey. ⸻ Question → How are you engineering context into your LLM workflows today? What's your best practice for context management for your AI code assistants? And where does it still break down? #AIEngineering #ContextEngineering #SoftwareDevelopment #DevTools

Most people think agents fail because the model “didn’t understand the request.” 🙈 In reality, many agents fail because their context window gets flooded with old tool results — JSON blobs, search results, code edits, logs… all quietly accumulating in the background 💥📚 Anthropic introduced a very clever solution: Context Editing. It automatically clears older tool results (and thinking blocks) once the prompt grows too large. Super elegant — but limited to Claude. So we brought that idea into LangChain… and made it model-agnostic 🔓⚙️ That means you now get Anthropic-style context management with OpenAI, Google Gemini, xAI, local models etc. The new Context Editing Middleware lets you: 🧹 Automatically clear older tool outputs 🔧 Keep only the recent results you care about 🚫 Exclude specific tools from being cleared 📉 Reduce token usage dramatically 🧠 Extend long-running agent workflows without blowing up context windows I walk through the concept, show how Anthropic’s strategy works under the hood, and explain how we adapted it for LangChainJS in today’s video 👇 🎥 https://lnkd.in/gx66R5YE 📚 https://lnkd.in/gejh5GFP If you’re building tool-heavy agents, this is a big quality-of-life improvement — and it works across your entire model stack. #LangChain #AI #agents #openai #anthropic #contextengineering