LLM Development Principles

𝗟𝗟𝗠 𝘃𝘀. 𝗥𝗔𝗚 𝘃𝘀. 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 𝘃𝘀. 𝗔𝗴𝗲𝗻𝘁 𝘃𝘀. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 — 𝗪𝗵𝗲𝗻 𝘁𝗼 𝘂𝘀𝗲 𝘄𝗵𝗮𝘁 I keep getting one question from teams building with GenAI: Which approach should we choose? This one-pager visual breaks down the trade-offs. Below is the practical guide I use on real projects. 𝟭) 𝗟𝗟𝗠 What it is: Prompt → model → answer. Use when: General knowledge, ideation, drafting, small utilities. Watch out for: Hallucinations on domain-specific facts; limited to model’s pretraining. 𝟮) 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻) What it is: Query → retrieve context from a knowledge base → feed context + query to LLM → grounded answer. Model weights don’t change. Use when: You have proprietary docs, policies, catalogs, tickets, or logs that change frequently. Benefits: Lower cost than training, auditable sources, fast updates. Key tips: Good chunking, embeddings, metadata, and re-ranking determine quality more than the LLM choice. 𝟯) 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 What it is: Train the model on input→output pairs to change its weights (LLM → LLM′). Use when: You need consistent style, domain tone, or task-specific behavior (classification, templated replies, structured outputs). Benefits: Lower prompt complexity, stable behavior, smaller inference tokens. Caveats: Needs clean, labeled data; versioning and evaluation are critical. 𝟰) 𝗔𝗴𝗲𝗻𝘁 What it is: LLM + memory + tools/APIs with a think → act → observe loop. Use when: Tasks require multi-step reasoning, tool use (search, SQL, APIs), or state over time. Examples: Troubleshooting flows, data enrichment, workflow automation. Risks: Loops, tool misuse, latency. Use guardrails, timeouts, and action limits. 𝟱) 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 (𝗠𝘂𝗹𝘁𝗶-𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺𝘀) What it is: Coordinated roles (planner, executor, critic) that plan → act → observe → learn from feedback. Use when: Complex processes with decomposition, review, and collaboration across specialized agents. Examples: Customer ops copilots, multi-step ETL with validation, enterprise workflows spanning multiple systems. Challenges: Orchestration, determinism, monitoring, and cost control. Metrics that matter Grounding: citation hit-rate, answer verifiability (RAG) Quality: task accuracy, pass@k, error rate Efficiency: latency, tokens, cost per resolution Safety: hallucination rate, tool misuse, policy violations Reliability: determinism, replayability, test coverage Design Tips: Start with RAG before touching fine-tuning; data beats weights early on. Keep prompts short; push knowledge to the retriever or the dataset. Add evaluation harnesses from day one (gold sets, unit tests for prompts/tools). Log everything: context windows, actions, failures, and human overrides. Treat agents like software: versioning, guardrails, circuit breakers, and audits.

If you’re an AI engineer trying to understand how reasoning actually works inside LLMs, this will help you connect the dots. Most large language models can generate. But reasoning models can decide. Traditional LLMs followed a straight line: Input → Predict → Output. No self-checking, no branching, no exploration. Reasoning models introduced structure, a way for models to explore multiple paths, score their own reasoning, and refine their answers. We started with Chain-of-Thought (CoT) reasoning, then extended to Tree-of-Thought (ToT) for branching, and now to Graph-based reasoning, where models connect, merge, or revisit partial thoughts before concluding. This evolution changes how LLMs solve problems. Instead of guessing the next token, they learn to search the reasoning space- exploring alternatives, evaluating confidence, and adapting dynamically. Different reasoning topologies serve different goals: • Chains for simple sequential reasoning • Trees for exploring multiple hypotheses • Graphs for revising and merging partial solutions Modern architectures (like OpenAI’s o-series reasoning models, Anthropic’s Claude reasoning stack, DeepSeek R series and DeepMind’s AlphaReasoning experiments) use this idea under the hood. They don’t just generate answers, they navigate reasoning trajectories, using adaptive depth-first or breadth-first exploration, depending on task uncertainty. Why this matters? • It reduces hallucinations by verifying intermediate steps • It improves interpretability since we can visualize reasoning paths • It boosts reliability for complex tasks like planning, coding, or tool orchestration The next phase of LLM development won’t be about more parameters, it’ll be about better reasoning architectures: topologies that can branch, score, and self-correct. I’ll be doing a deep dive on reasoning models soon on my Substack- exploring architectures, training approaches, and practical applications for engineers. If you haven’t subscribed yet, make sure you do: https://lnkd.in/dpBNr6Jg ♻️ Share this with your network 🔔 Follow along for more data science & AI insights

In the last few months, I have explored LLM-based code generation, comparing Zero-Shot to multiple types of Agentic approaches. The approach you choose can make all the difference in the quality of the generated code. Zero-Shot vs. Agentic Approaches: What's the Difference? ⭐ Zero-Shot Code Generation is straightforward: you provide a prompt, and the LLM generates code in a single pass. This can be useful for simple tasks but often results in basic code that may miss nuances, optimizations, or specific requirements. ⭐ Agentic Approach takes it further by leveraging LLMs in an iterative loop. Here, different agents are tasked with improving the code based on specific guidelines—like performance optimization, consistency, and error handling—ensuring a higher-quality, more robust output. Let’s look at a quick Zero-Shot example, a basic file management function. Below is a simple function that appends text to a file: def append_to_file(file_path, text_to_append): try: with open(file_path, 'a') as file: file.write(text_to_append + '\n') print("Text successfully appended to the file.") except Exception as e: print(f"An error occurred: {e}") This is an OK start, but it’s basic—it lacks validation, proper error handling, thread safety, and consistency across different use cases. Using an agentic approach, we have a Developer Lead Agent that coordinates a team of agents: The Developer Agent generates code, passes it to a Code Review Agent that checks for potential issues or missing best practices, and coordinates improvements with a Performance Agent to optimize it for speed. At the same time, a Security Agent ensures it’s safe from vulnerabilities. Finally, a Team Standards Agent can refine it to adhere to team standards. This process can be iterated any number of times until the Code Review Agent has no further suggestions. The resulting code will evolve to handle multiple threads, manage file locks across processes, batch writes to reduce I/O, and align with coding standards. Through this agentic process, we move from basic functionality to a more sophisticated, production-ready solution. An agentic approach reflects how we can harness the power of LLMs iteratively, bringing human-like collaboration and review processes to code generation. It’s not just about writing code; it's about continuously improving it to meet evolving requirements, ensuring consistency, quality, and performance. How are you using LLMs in your development workflows? Let's discuss!

Every week a new “best” large language model makes headlines, but in practice there is no single model that can do it all. Some are strong at reasoning, others are faster or more cost-efficient. Benchmarks can give a snapshot, but they rarely reflect how a model will perform on your real-world problems. The best way to evaluate is to test directly against your toughest use cases. A recent O’Reilly piece made a key point: success with LLMs is not about picking the flashiest model. It is about system design. That often means combining multiple models, using lightweight ones for simple tasks, stronger ones for deep reasoning, and others for validation. Open-weight models offer control and customization, while closed APIs bring cutting-edge performance with less flexibility. The real advantage comes from knowing how to mix and match these tools to create a system that is cost-effective, reliable, and fit for purpose. https://lnkd.in/epqdSKQh

Fascinating new research comparing Long Context LLMs vs RAG approaches! A comprehensive study by researchers from Nanyang Technological University Singapore and Fudan University reveals key insights into how these technologies perform across different scenarios. After analyzing 12 QA datasets with over 19,000 questions, here's what they discovered: Key Technical Findings: - Long Context (LC) models excel at processing Wikipedia articles and stories, achieving 56.3% accuracy compared to RAG's 49.0% - RAG shows superior performance in dialogue-based contexts and fragmented information - RAPTOR, a hierarchical tree-based retrieval system, outperformed traditional chunk-based and index-based retrievers with 38.5% accuracy Under the Hood: The study implements a novel three-phase evaluation framework: 1. Empirical retriever assessment across multiple architectures 2. Direct LC vs RAG comparison using filtered datasets 3. Granular analysis of performance patterns across different question types and knowledge sources Most interesting finding: RAG exclusively answered 10% of questions that LC couldn't handle, suggesting these approaches are complementary rather than competitive. The research team also introduced an innovative question filtering methodology to ensure fair comparison by removing queries answerable through parametric knowledge alone. This work significantly advances our understanding of when to use each approach in production systems. A must-read for anyone working with LLMs or building RAG systems!

Last week the bigs news from Anthropic was the release of Opus 4.5, but they dropped another release in parallel that fell under the radar which I'm dubbing "the workflow killer". For the past few years since OpenAI introduced the concept of "function calling" to the ChatCompletions API, the primary way LLM agents interact with the world is via requesting "tool calls" via JSON. For example, you tell the LLM that it has access to a "get_weather(...)" tool, and if the agent decides it needs to check the weather, it will output a request to the "get_weather" tool with args (e.g. "city=SanFrancisco"). However, this is quite limiting - what if the agent wants to get the weather for 100 cities in parallel? What if the agent want to iterate through a list of 100 cities stored in a CSV file, and get the weather for each one? In the default agent setup, each of these examples would require 100 LLM calls to generate 100 tool calls, which is clearly a waste. That's where programmatic tool calling comes in: the agent can simply write code (e.g. Python) that invokes the tools specified by the developer. For example, it can write a Python script to iterate over a CSV file and call the "get_weather" tool 100 times - the key difference is that the agent can specify this tool calling sequence via a single code block, requiring a single LLM call (not 100). Why is this a "workflow killer"? The benefit of "agentic workflows" over fully autonomous agents is that workflows can contain complex sequences of specific tools which are easiest specified in code, rather than human instructions. With programmatic tool calling, the agents can now write the workflows themselves! An important side note: the concepts in Anthropic "programmatic tool calling" feature aren't new, they've existed for a while: see Apple's CodeAct paper, or Cloudflare's Code Mode MCP feature. But it's the first time the idea has been implemented as a first-party API primitive by a major frontier lab. If you want to try out programmatic tool calling, we just added support for it in the Letta API, so you can experiment with the feature on any model (not just Claude)!

𝐋𝐋𝐌 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐯𝐬 𝐅𝐢𝐧𝐞-𝐓𝐮𝐧𝐢𝐧𝐠 𝐯𝐬 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞: 𝐓𝐡𝐞 𝟑 𝐏𝐢𝐥𝐥𝐚𝐫𝐬 𝐨𝐟 𝐀𝐈 𝐀𝐠𝐞𝐧𝐭 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭 Large Language Models (LLMs) do not just appear magically ready to answer your questions. Their journey to becoming smart assistants involves three very different stages and each one matters when building AI agents. 𝟏. 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 This is the childhood and education phase. The model is exposed to massive datasets books, articles, code, and more to learn general language patterns. It’s expensive, time-consuming, and usually done only by big AI labs. 𝟐. 𝐅𝐢𝐧𝐞-𝐓𝐮𝐧𝐢𝐧𝐠 This is the specialisation phase. Here, a pre-trained model is adapted to a specific domain or task. For example: teaching a general model to excel in medical advice, legal reasoning, or customer support scripts. It’s faster and cheaper than full training, but still needs good-quality, domain-specific data. 𝟑. 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 This is the job performance phase. It’s when the model is put to work generating answers, solving problems, and interacting with users in real time. Efficiency here depends on how well the model was trained, fine-tuned, and optimised. 𝐖𝐡𝐲 𝐝𝐨𝐞𝐬 𝐭𝐡𝐢𝐬 𝐦𝐚𝐭𝐭𝐞𝐫? 𝐖𝐡𝐞𝐧 𝐲𝐨𝐮 𝐚𝐫𝐞 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐀𝐈 𝐚𝐠𝐞𝐧𝐭𝐬, 𝐲𝐨𝐮 𝐧𝐞𝐞𝐝 𝐭𝐨 𝐤𝐧𝐨𝐰 𝐰𝐡𝐞𝐫𝐞 𝐭𝐨 𝐢𝐧𝐯𝐞𝐬𝐭 𝐲𝐨𝐮𝐫 𝐞𝐟𝐟𝐨𝐫𝐭: - Full training is overkill unless you are creating something from scratch. - Fine-tuning helps your agent become an expert in your domain. - Optimising inference ensures it is fast, responsive, and cost-effective for users. 𝐈𝐦𝐩𝐚𝐜𝐭 𝐨𝐧 ��𝐈 𝐀𝐠𝐞𝐧𝐭 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: - Poor training → weak foundation - No fine-tuning → generic, unhelpful responses - Slow inference → bad user experience Balancing these stages is what turns an AI agent from “𝐣𝐮𝐬𝐭 𝐚𝐧𝐨𝐭𝐡𝐞𝐫 𝐜𝐡𝐚𝐭𝐛𝐨𝐭” into a specialised, high-performance assistant. 𝐈𝐟 𝐲𝐨𝐮 𝐡𝐚𝐝 𝐭𝐨 𝐜𝐡𝐨𝐨𝐬𝐞 𝐨𝐧𝐥𝐲 𝐨𝐧𝐞 𝐭𝐨 𝐢𝐧𝐯𝐞𝐬𝐭 𝐢𝐧 𝐟𝐨𝐫 𝐲𝐨𝐮𝐫 𝐀𝐈 𝐚𝐠𝐞𝐧𝐭 𝐫𝐢𝐠𝐡𝐭 𝐧𝐨𝐰 𝐛𝐞𝐭𝐭𝐞𝐫 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐝𝐚𝐭𝐚, 𝐝𝐞𝐞𝐩𝐞𝐫 𝐟𝐢𝐧𝐞-𝐭𝐮𝐧𝐢𝐧𝐠, 𝐨𝐫 𝐟𝐚𝐬𝐭𝐞𝐫 𝐢𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞, 𝐰𝐡𝐢𝐜𝐡 𝐰𝐨𝐮𝐥𝐝 𝐲𝐨𝐮 𝐩𝐫𝐢𝐨𝐫𝐢𝐭𝐢𝐬𝐞?

Been experimenting a lot with AI coding tools (Lovable, Cursor, etc.) and had a bit of a realization. While code generation saves time, the more profound shift might be using LLMs as functional components within our code. Plunging inference costs make it viable to replace chunks of complex, brittle code (think intricate parsing or rule engines) with a validated LLM call. It's a different way of thinking about where complexity should live. I wrote about this "LLM as function" idea, why it's possible now, and included a practical Python date parsing example using Instructor AI. Would love to hear if others are exploring this too.

“𝐒𝐨𝐫𝐫𝐲, 𝐰𝐞’𝐫𝐞 𝐤𝐞𝐞𝐩𝐢𝐧𝐠 𝐲𝐨𝐮𝐫 𝐀𝐓𝐌 𝐜𝐚𝐫𝐝, 𝐲𝐨𝐮 𝐚𝐭𝐭𝐞𝐦𝐩𝐭𝐞𝐝 𝐲𝐨𝐮𝐫 𝐏𝐈𝐍 𝐭𝐨𝐨 𝐦𝐚𝐧𝐲 𝐭𝐢𝐦𝐞𝐬!” 𝐓𝐡𝐞 𝐑𝐢𝐬𝐞 𝐨𝐟 𝐀𝐠𝐞𝐧𝐭-𝐍𝐚𝐭𝐢𝐯𝐞 𝐀𝐏𝐈𝐬: 𝐇𝐨𝐰 𝐂𝐥𝐨𝐮𝐝𝐟𝐥𝐚𝐫𝐞’𝐬 𝐒𝐃𝐊 𝐏𝐨𝐢𝐧𝐭𝐬 𝐭𝐨 𝐭𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞 For those of you of a certain vintage, you share my PTSD from watching some ATM, near campus, swallow up your card on your way to a Friday night party!! (Note: my balance was usually $0, so the ATM probably did me a favor saving me from further embarrassment on some date). In any event, the frustration from making multiple attempts to an ATM is how AI agents are feeling trying to use current APIs. In earlier posts, I’ve talked about how current APIs were built for humans, not agents as they expect precise, deterministic inputs rather than probabilistic, context-rich reasoning from LLMs (read: hallucinations). 𝐖𝐡𝐲 𝐈𝐬 𝐓𝐡𝐢𝐬 𝐚 𝐏𝐫𝐨𝐛𝐥𝐞𝐦? APIs rely on rigid, structured schemas to ensure exact matches when writing, retrieving, updating, or deleting data (CRUD). These schemas are usually represented as JSON objects with specific parameters. 𝐋𝐋𝐌𝐬, 𝐡𝐨𝐰𝐞𝐯𝐞𝐫, 𝐬𝐭𝐫𝐮𝐠𝐠𝐥𝐞 𝐭𝐨 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐭𝐡𝐞𝐬𝐞 𝐉𝐒𝐎𝐍 𝐨𝐛𝐣𝐞𝐜𝐭𝐬 𝐜𝐨𝐫𝐫𝐞𝐜𝐭𝐥𝐲. When agents use LLMs to construct the precise schema needed for an API, the model often hallucinates, mislabels fields, or omits critical parameters which leads to failed or inaccurate transactions. 𝐂𝐥𝐨𝐮𝐝𝐟𝐥𝐚𝐫𝐞’𝐬 𝐒𝐨𝐥𝐮𝐭𝐢𝐨𝐧: 𝐋𝐞𝐭 𝐭𝐡𝐞 𝐋𝐋𝐌 𝐂𝐨𝐝𝐞 That’s why Cloudflare’s new Agents SDK is fascinating. Instead of relying on structured JSON “function calls,” it lets LLMs generate actual TypeScript code, which is then executed safely in low-latency V8 environments. 𝐖𝐡𝐲 𝐓𝐡𝐢𝐬 𝐈𝐬 𝐚 𝐁𝐢𝐠 𝐃𝐞𝐚𝐥 𝟏-“𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠” 𝐯𝐬. 𝐆𝐮𝐞𝐬𝐬𝐢𝐧𝐠 LLMs are much better at iterating through code they’ve written (reading error messages, fixing bugs, and retrying) than they are at perfectly generating a static JSON schema APIs. So Cloudflare’s SDK lets the LLM rely on one of its current strengths and generate TypeScript code to interact with APIs dynamically, instead of making a fragile one-shot JSON call. 𝟐-“𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠” 𝐒𝐚𝐧𝐝𝐛𝐨𝐱𝐞𝐬 These lightweight/low-latency V8 isolates act like tiny, secure test labs where the LLM can run and refine its code in real time. This gives agents multiple “attempts” to get the interaction right and reduce failed calls. This is a major architectural step forward that bridges the gap between probabilistic reasoning and deterministic execution where agents move from guessing what an API expects to programmatically reasoning through the call itself. This offers a glimpse into the next phase of agentic computing, where APIs, SDKs, and infrastructure are natively designed for machines talking to machines, not humans typing into web forms. https://lnkd.in/e7E2EvXt #AI #LLM #AIAgents #AgenticComputing #Cloudflare #DeveloperTools #API

❌ "𝗝𝘂𝘀𝘁 𝘂𝘀𝗲 𝗖𝗵𝗮𝘁𝗚𝗣𝗧" 𝗶𝘀 𝘁𝗲𝗿𝗿𝗶𝗯𝗹𝗲 𝗮𝗱𝘃𝗶𝗰𝗲. Here's what most AI & Automation leaders get wrong about LLMs: They're building their entire AI infrastructure around ONE or TWO models. The reality? There is no single "best LLM." The top models swap positions every few months, and each has unique strengths and costly blindspots. I analyzed the 6 frontier models driving enterprise AI today. Here's what I found: 𝟭. 𝗚𝗲𝗺𝗶𝗻𝗶 (𝟯 𝗣𝗿𝗼/𝗨𝗹𝘁𝗿𝗮) ✓ Superior reasoning and multimodality ✓ Excels at agentic workflows ✗ Not useful for writing tasks 𝟮. 𝗖𝗵𝗮𝘁𝗚𝗣𝗧 (𝗚����𝗧-𝟱) ✓ Most reliable all-around ✓ Mature ecosystem ✗ A lot prompt-dependent 𝟯. 𝗖𝗹𝗮𝘂𝗱𝗲 (𝟰.𝟱 𝗦𝗼𝗻𝗻𝗲𝘁/𝗢𝗽𝘂𝘀) ✓ Industry leader in coding & debugging ✓ Enterprise-grade safety ✗ Opus is very expensive 𝟰. 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸 (𝗩𝟯.𝟮-𝗘𝘅𝗽) ✓ Great cost-efficiency ✓ Top-tier coding and math ✗ Less mature ecosystem 𝟱. 𝗚𝗿𝗼𝗸 (𝟰/𝟰.𝟭) ✓ Real-time data access ✓ High-speed querying ✗ Limited free access 𝟲. 𝗞𝗶𝗺𝗶 𝗔𝗜 (𝗞𝟮 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴) ✓ Massive context windows ✓ Superior long document analysis ✗ Chinese market focus The winning strategy isn't picking one. It's orchestration. Here's the playbook: → Stop hardcoding single-vendor APIs → Route code writing & reviews to Claude → Send agentic & multimodal workflows to Gemini → Use DeepSeek for cost-effective baseline tasks → Build multi-step workflows, not one-shot prompts 𝗧𝗵𝗲 𝗯𝗼𝘁𝘁𝗼𝗺 𝗹𝗶𝗻𝗲? Your competitive advantage isn't choosing the "best" model. It's building orchestration systems that route intelligently across all of them. The future of enterprise automation is agentic systems that manage your LLM landscape for you. What's the LLM strategy that's working for you? ---- 🎯 Follow for Agentic AI, Gen AI & RPA trends: https://lnkd.in/gFwv7QiX Repost if this helped you see the shift ♻️