LLM Strategies for Human-Level Performance

In the last three months alone, over ten papers outlining novel prompting techniques were published, boosting LLMs’ performance by a substantial margin. Two weeks ago, a groundbreaking paper from Microsoft demonstrated how a well-prompted GPT-4 outperforms Google’s Med-PaLM 2, a specialized medical model, solely through sophisticated prompting techniques. Yet, while our X and LinkedIn feeds buzz with ‘secret prompting tips’, a definitive, research-backed guide aggregating these advanced prompting strategies is hard to come by. This gap prevents LLM developers and everyday users from harnessing these novel frameworks to enhance performance and achieve more accurate results. https://lnkd.in/g7_6eP6y In this AI Tidbits Deep Dive, I outline six of the best and recent prompting methods: (1) EmotionPrompt - inspired by human psychology, this method utilizes emotional stimuli in prompts to gain performance enhancements (2) Optimization by PROmpting (OPRO) - a DeepMind innovation that refines prompts automatically, surpassing human-crafted ones. This paper discovered the “Take a deep breath” instruction that improved LLMs’ performance by 9%. (3) Chain-of-Verification (CoVe) - Meta's novel four-step prompting process that drastically reduces hallucinations and improves factual accuracy (4) System 2 Attention (S2A) - also from Meta, a prompting method that filters out irrelevant details prior to querying the LLM (5) Step-Back Prompting - encouraging LLMs to abstract queries for enhanced reasoning (6) Rephrase and Respond (RaR) - UCLA's method that lets LLMs rephrase queries for better comprehension and response accuracy Understanding the spectrum of available prompting strategies and how to apply them in your app can mean the difference between a production-ready app and a nascent project with untapped potential. Full blog post https://lnkd.in/g7_6eP6y

If you’re building LLM applications today, reasoning is where the real leverage lies. And yet, I see a lot of engineers still treating LLM outputs as a single-shot black box. LLMs can reason, but only if you give them the right scaffolding and the right post-training. Here’s a mental model I’ve been using to think about LLM reasoning methods (see chart below): ✅ Inference-time reasoning methods: These are techniques that can be applied at inference time, without needing to retrain your model: → Tree of Thoughts (ToT), search through reasoning paths → Chain of Thought (CoT) prompting, prompt models to generate intermediate reasoning steps → Reasoning + Acting, use tools or function calls during reasoning → Self-feedback, prompt the model to critique and refine its own output → Episodic Memory Agents, maintain a memory buffer to improve multi-step reasoning → Self-consistency, sample multiple reasoning paths and select the most consistent answer ✅ Training-time enhancements: Where things get really powerful is when you post-train your model to improve reasoning, using human annotation or policy optimization: → Use Preference pairs and Reward Models to tune for better reasoning (RFT, Proximal PO, KL Regularization) → Apply RLHF, PPO + KL, Rejection Sampling + SFT, Advantage Estimation, and other advanced techniques to guide the model’s policy → Leverage multiple paths, offline trajectories, and expert demonstrations to expose the model to rich reasoning signals during training Here are my 2 cents 🫰 If you want production-grade LLM reasoning, you’ll need both, → Smart inference-time scaffolds to boost reasoning without slowing latency too much → Carefully tuned post-training loops to align the model’s policy with high-quality reasoning patterns → We’re also seeing increasing use of Direct Preference Optimization (DPO) and reference-free grading to further improve reasoning quality and stability. I’m seeing more and more teams combine both strategies, and the gap between "vanilla prompting" and "optimized reasoning loops" is only getting wider. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

LLMs are no longer just fancy autocomplete engines. We’re seeing a clear shift—from single-shot prompting to techniques that mimic 𝗮𝗴𝗲𝗻𝗰𝘆: reasoning, retrieving, taking action, and even coordinating across steps. In this visual, I’ve laid out five core prompting strategies: - 𝗥𝗔𝗚 – Brings in external knowledge, enhancing factual accuracy   - 𝗥𝗲𝗔𝗰𝘁 – Enables reasoning 𝗮𝗻𝗱 acting, the essence of agentic behavior   - 𝗗𝗦𝗣 – Adds directional hints through policy models   - 𝗧𝗼𝗧 (𝗧𝗿𝗲𝗲-𝗼𝗳-𝗧𝗵𝗼𝘂𝗴𝗵𝘁) – Simulates branching reasoning paths, like a mini debate inside the LLM   - 𝗖𝗼𝗧 (𝗖𝗵𝗮𝗶𝗻-𝗼𝗳-𝗧𝗵𝗼𝘂𝗴𝗵𝘁) – Breaks down complex thinking into step-by-step logic While not all of these are fully agentic on their own, techniques like 𝗥𝗲𝗔𝗰𝘁 and 𝗧𝗼𝗧 are clear stepping stones to 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 — where autonomous agents can 𝗿𝗲𝗮𝘀𝗼𝗻, 𝗽𝗹𝗮𝗻, 𝗮𝗻𝗱 𝗶𝗻𝘁𝗲𝗿𝗮𝗰𝘁 𝘄𝗶𝘁𝗵 𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁𝘀. The big picture?  We’re slowly moving from "𝘱𝘳𝘰𝘮𝘱𝘵 𝘦𝘯𝘨𝘪𝘯𝘦𝘦𝘳𝘪𝘯𝘨" to "𝘤𝘰𝘨𝘯𝘪𝘵𝘪𝘷𝘦 𝘢𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵𝘶𝘳𝘦 𝘥𝘦𝘴𝘪𝘨𝘯." And that’s where the real innovation lies.

🧠 We just implemented the "third paradigm" for LLM learning - and the results are promising. Most of us know that leading AI applications like ChatGPT, Claude, and Grok achieve their impressive performance partly through sophisticated system prompts containing detailed reasoning strategies and problem-solving frameworks. Yet most developers and researchers work with basic prompts, missing out on these performance gains. 🚀 Introducing System Prompt Learning (SPL) Building on Andrej Karpathy's vision of a "third paradigm" for LLM learning, SPL enables models to automatically learn and improve problem-solving strategies through experience, rather than relying solely on pre-training or fine-tuning. ⚙️ How it works: 🔍 Automatically classifies incoming problems into 16 types 📚 Builds a persistent database of effective solving strategies 🎯 Selects the most relevant strategies for each new query 📊 Evaluates strategy effectiveness and refines them over time 👁️ Maintains human-readable, inspectable knowledge 📈 Results across mathematical benchmarks: OptILLMBench: 61% → 65% (+4%) MATH-500: 85% → 85.6% (+0.6%) Arena Hard: 29% → 37.6% (+8.6%) AIME24: 23.33% → 30% (+6.67%) After just 500 training queries, our system developed 129 strategies, refined 97 existing ones, and achieved 346 successful problem resolutions. ✨ What makes this approach unique: 🔄 Cumulative learning that improves over time 📖 Transparent, human-readable strategies 🔌 Works with any OpenAI-compatible API 🔗 Can be combined with other optimization techniques ⚡ Operates in both inference and learning modes 📝 Example learned strategy for word problems: 1. Understand: Read carefully, identify unknowns 2. Plan: Define variables, write equations 3. Solve: Step-by-step with units 4. Verify: Check reasonableness This represents early progress toward AI systems that genuinely learn from experience in a transparent, interpretable way - moving beyond static models to adaptive systems that develop expertise through practice. 🛠️ Implementation: SPL is available as an open-source plugin in optillm, our inference optimization proxy. Simple integration by adding "spl-" prefix to your model name. The implications extend beyond current capabilities - imagine domain-specific expertise development, collaborative strategy sharing, and human expert contributions to AI reasoning frameworks. 💭 What are your thoughts on LLMs learning from their own experience? Have you experimented with advanced system prompting in your work? #ArtificialIntelligence #MachineLearning #LLM #OpenSource #TechInnovation #ProblemSolving #AI #Research

🤔 What if, instead of using prompts, you could fine-tune LLMs to incorporate self-feedback and improvement mechanisms more effectively? Self-feedback and improvement have been shown to be highly beneficial for LLMs and agents, allowing them to reflect on their behavior and reasoning and correct their mistakes as more computational resources or interactions become available. The authors mention that frequently used test-time methods like prompt tuning and few-shot learning that are used for self-improvement, often fail to enable models to correct their mistakes in complex reasoning tasks. ⛳ The paper introduces RISE: Recursive Introspection, an approach to improve LLMs by teaching them how to introspect and improve their responses iteratively. ⛳ RISE leverages principles from online imitation learning and reinforcement learning to develop a self-improvement mechanism within LLMs. By treating each prompt as part of a multi-turn Markov decision process (MDP), RISE allows models to learn from their previous attempts and refine their answers over multiple turns, ultimately improving their problem-solving capabilities. ⛳It models the fine-tuning process as a multi-turn Markov decision process, where the initial state is the prompt, and subsequent states involve recursive improvements. ⛳It employs a reward-weighted regression (RWR) objective to learn from both high- and low-quality rollouts, enabling models to improve over turns. The approach uses data generated by the learner itself or more capable models to supervise improvements iteratively. RISE significantly improves the performance of LLMs like LLaMa2, LLaMa3, and Mistral on math reasoning tasks, outperforming single-turn strategies with the same computational resources. Link: https://lnkd.in/e2JDQr8M

Just finished reading an amazing book: AI Engineering by Chip Huyen. Here’s the quickest (and most agile) way to build LLM products: 1. Define your product goals Pick a small, very clear problem to solve (unless you're building a general chatbot). Identify use case and business objectives. Clarify user needs and domain requirements. 2. Select the foundation model Don’t waste time training your own at the start. Evaluate models for domain relevance, task capability, cost, and privacy. Decide on open source vs. proprietary options. 3. Gather and filter data Collect high-quality, relevant data. Remove bias, toxic content, and irrelevant domains. 4. Evaluate baseline model performance Use key metrics: cross-entropy, perplexity, accuracy, semantic similarity. Set up evaluation benchmarks and rubrics. 5. Adapt the model for your task Start with prompt engineering (quick, cost-effective, doesn’t change model weights): craft detailed instructions, provide examples, and specify output formats. Use RAG if your application needs strong grounding and frequently updated factual data: integrate external data sources for richer context. Prompt-tuning isn’t a bad idea either. Still getting hallucinations? Try “abstention”—having the model say “I don’t know” instead of guessing. 6. Fine-tune (only if you have a strong case for it) Train on domain/task-specific data for better performance. Use model distillation for cost-efficient deployment. 7. Implement safety and robustness Protect against prompt injection, jailbreaks, and extraction attacks. Add safety guardrails and monitor for security risks. 8. Build memory and context systems Design short-term and long-term memory (context windows, external databases). Enable continuity across user sessions. 9. Monitor and maintain Continuously track model performance, drift, evaluation metrics, business impact, token usage, etc. Update the model, prompts, and data based on user feedback and changing requirements. Observability is key! 10. Test, Test, Test! Use LLM judges, human-in-the-loop strategies; iterate in small cycles. A/B test in small iterations: see what breaks, patch, and move on. A simple GUI or CLI wrapper is just fine for your MVP. Keep scope under control—LLM products can be tempting to expand, but restraint is crucial! Fastest way: Build an LLM optimized for a single use case first. Once that works, adding new use cases becomes much easier. https://lnkd.in/ghuHNP7t Summary video here -> https://lnkd.in/g6fPsqUR Chip Huyen, #AiEngineering #LLM #GenAI #Oreilly #ContinuousLEarning #ProductManagersinAI