Jonathan Dodson

Just getting started with LLMs? Don’t let the jargon slow you down. This 5-part LLM Glossary is your shortcut to mastering the fundamentals👇 1️⃣ Model Types • Foundation Model: Pretrained on large-scale, diverse datasets to learn general capabilities. • Instruction-Tuned Model: Further trained to follow user instructions precisely. • Multi-modal Model: Handles multiple input/output formats like text, images, and audio. • Reasoning Model: Optimized for logic, problem-solving, and step-by-step thinking. • Small Language Model (SLM): Lightweight models for fast, efficient, on-device tasks. 2️⃣ Training & Fine-Tuning • Pretraining: Initial learning phase using massive, diverse data sources. • RLHF (Reinforcement Learning with Human Feedback): Aligns model behavior with human preferences. • DPO (Direct Preference Optimization): Uses ranked preferences instead of rewards for training. • Synthetic Data: AI-generated data used to supplement real datasets. • Fine-Tuning: Targeted retraining on specific domains or tasks. • LoRA / QLoRA: Techniques for efficient low-resource fine-tuning. • Guardrails: Rules or filters applied to enforce safety, ethics, and compliance. 3️⃣ Prompt Engineering • System/User Prompts: Define model behavior and user context in a session. • Chain-of-Thought (CoT): Prompts that guide the model to reason step-by-step. • Few-Shot / Zero-Shot Learning: Demonstrating tasks with few or no examples. • Prompt Tuning: Optimizing prompts via training for specific outcomes. • Context Window: The total number of tokens the model can "remember" per interaction. 4️⃣ Inference & Generation • Temperature: Adjusts output randomness; lower values are more deterministic. • Max Tokens: Limits the number of tokens in a generated response. • Seed: Controls reproducibility of outputs. • Latency: Time taken by the model to respond. • Hallucination: When the model generates factually incorrect but plausible-sounding information. 5️⃣ Retrieval-Augmented Generation (RAG) • Retrieval: Fetching relevant data from external sources before generation. • Semantic Search: Search based on meaning, not just keywords. • Embeddings: Vector representations of text used for similarity matching. • Chunks: Dividing documents into smaller segments for better retrieval. • Vector Databases (VectorDBs): Stores embeddings for fast and accurate search. • Reranking: Rescoring retrieved documents to prioritize relevance. • Indexing: Structuring data for efficient retrieval during generation. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

232

36 Comments

LLMs alone aren’t enough to scale agents in the enterprise. What makes agents reliable isn’t just model quality—it’s the system around them. We’ve learned that high-quality search is foundational not only for grounding agents in the enterprise context but also for mapping process knowledge, personalizing agents, and scaling agents usage of enterprise tools and workflows. Here’s what that system looks like at Glean: • Grounding agents in relevant, authoritative knowledge: Autonomous agents raise the bar — they can’t rely on outdated or incorrect knowledge. They need precise, task-aligned, enterprise-specific context. Knowledge bases alone aren’t enough — this challenge requires high-quality search. • Personalizing agents for each enterprise: Every organization works differently. We analyze usage to learn how work actually happens, then surface relevant, customer-specific examples to guide agent design. • Creating agents that understand how enterprises work: We’ve observed how Glean users progress from simple Assistant queries to more complex, agentic workflows. We’ve grouped these into distinct query classes — such as enumeration (“list all Jira tickets assigned to me”) or people-seeking (“who’s my manager’s manager?”). These classes help us guide agents in how to interpret intent and execute real work. • Scaling evaluation with LLM judges — For every query class, we build eval sets from real usage and design task-specific LLM judges. But building a judge isn’t enough — it must produce consistent results across similar inputs. Consistent, scalable evaluation is a precursor to agents tackling real work in the enterprise. We’re excited to keep pushing the boundaries of what agents can do at work. A huge credit to the team behind our agentic reasoning engine — every part of it is purpose-built for enterprise AI.

460

24 Comments

If you are getting started in AI engineering or trying to level up, eval engineering is a concept you need to understand early. Traditional AI evals break down because most teams rely on LLM-as-a-judge. Accuracy plateaus around 60–70%, costs scale fast, and teams end up evaluating only a small slice of production traffic. That means real failures slip through. Eval engineering treats evaluation as a system, not a metric. ✦ Start with LLM-as-a-judge to establish a baseline ✦ Involve subject matter experts to surface real failure modes ✦ Refine prompts and criteria using production data ✦ Train a small, fast judge model ✦ Deploy it for low-cost, low-latency evals across 100% of traffic ✦ Continuously feed failures back into the loop The goal is simple: move from sampled guesses to full observability, and from “it looks fine” to “we trust this in production.” Image inspiration: Galileo ✦✦✦ ♻️ Share this with your network to help them learn about latest data/AI concepts Follow me (Aishwarya Srinivasan) for more data science/ML/AI insights and subscribe to my Substack for more educational resources: https://lnkd.in/dpBNr6Jg

195

18 Comments

This Karpathy guy is _extremely_ credible, so when he says "AI agents are slop", it maybe means something. There's actually a lot more fascinating detail about gradient descent and log probs and LLM-as-judge in this podcast, definitely worth listening to if you're into the technical nuance, but let's stay at the high level here. The interesting part about this podcast is that it's really balanced. It's not "AGI is around the corner" and it's not "AI is just a bubble". His view is a balanced middle view that AI will change a lot of things but gradually, and despite the research progress so far there's a huge amount still to come that isn't easy at all. It's worth watching or reading if you're trying to understand and predict the next 5-10 years (and right now who isn't!), because Karpathy has forgotten more about AI than most of us will ever know. Here's some excerpts from what he said: - "there's some over-prediction going on in the industry" - "You can’t just tell them something and they’ll remember it. They’re cognitively lacking and it’s just not working." - "I feel like the problems are tractable, they’re surmountable, but they’re still difficult." - "The models have so many cognitive deficits." - "It’s a total mess. It’s just not net useful. I can go in, I can clean it up, but it’s not net useful." - "They’re not very good at code that has never been written before" - "You’re not getting the richness and the diversity and the entropy from these models as you would get from humans." - "Look at the average example in the training set: factual mistakes, errors, nonsensical things." - "So this thing which is “general”, which should be able to do any knowledge work, is just overwhelmingly doing only coding." - "This assumption of suddenly having a completely intelligent, fully flexible, fully general human in a box, and we can dispense it at arbitrary problems in society, I don’t think that we will have this discrete change." - "For some kinds of tasks and jobs and so on, there’s a very large demo-to-product gap where the demo is very easy, but the product is very hard." - "I’m only sounding pessimistic because when I go on my Twitter timeline, I see all this stuff that makes no sense to me." But also: - "I see AI as fundamentally an extension of computing in a pretty fundamental way" - "Everything is gradually being automated and has been for hundreds of years." - "They’re going to get better, and it’s going to be wonderful." - "This is a miraculous technology that didn’t exist. There’s going to be a huge amount of demand, as we see the demand in ChatGPT already and so on."

95

49 Comments

Anthropic just cut off xAI from using Claude and this was inevitable. Last week, Anthropic reportedly blocked xAI’s access to Claude after discovering the team was using it via Cursor to speed up internal development. Anthropic’s terms are clear: you can’t use Claude to build or train a competing AI system. xAI crossed that line. Internally, xAI acknowledged the block, framing it as a short-term productivity hit and a push to accelerate their own tooling. This isn’t new behavior. Anthropic has taken similar action before, including restricting OpenAI and limiting Windsurf over competitive concerns. Why this matters: This episode says less about policy violations and more about where power has shifted. Claude, and especially Claude Code, has become critical infrastructure for AI development itself. When even rival frontier labs rely on a competitor’s model to build faster, you know that model is having a moment. But that also guarantees friction. We’re entering a phase where: 🔹 Models are competitors 🔹 Tools are dependencies 🔹 And “coopetition” gets very messy very fast This won’t be the last cutoff. The real battle now isn’t just whose model is best, it’s who controls the tools everyone else needs to build.

133

18 Comments

Whether you believe Databricks is worth $100B or not, and whether you like what it offers or not, here’s the reality: Databricks has pushed all the major players to deliver more value out of the box. The days of stitching together a thousand different cloud tools are fading, except for when you need a very specialized tool. Databricks’ strategy of bundling many affordable tools in one platform has been a major driver of this shift. Again, whether you like them or not, every other platform is now fighting for market share by trying to look more like Databricks. This is good for customers.

380

27 Comments

The best part of being at #Dreamforce is talking to many (MANY!) customers and executives about their biggest challenges in adopting AI agents. The same themes keep surfacing: figuring out where agents actually add value, integrating them into existing workflows, and making sure teams trust the output. My take on it (which has been true for most AI applications for years): 1️⃣ Start with a contained workflow. Don’t “launch an AI agent” . Instead, automate one painful, repetitive process, like summarizing customer calls or drafting support emails. 2️⃣ Focus on clean, connected data before fancy models. Invest in data context. Give your agents access to the right internal docs, CRM notes, and policies and make sure your data is clean, high-quality, and connected. That’s where the real power of agents comes from. 3️⃣ Build trust loops. Let humans review early outputs, collect feedback, and refine prompts and policies before scaling. Focus on the handoff between agents and humans, not replacement. It’s been fascinating to see what’s working (and what’s not) for our customers across many industries. How are you approaching AI agent pilots in your organization? how can we/I help? #salesforce #dreamforce #aileadership #agenticenterprise

111

3 Comments

As many organizations are starting to discover the requirements to build a successful and capable agent or agentic workflow run far deeper than just hooking up an LLM. This thoughtful perspective from Adrian Bridgwater unpacks the need to treat data integration like infrastructure within a 'deep iPaaS' layer to fully harness the power of AI. Beyond the fact it's always exciting (to me!) to see Tray mentioned in publications like this, it is a topic that has a major impact on ultimately how successful AI implementation is. https://lnkd.in/eaZ5Hkwg

41

2 Comments

It has arrived! Search Query Performance (SQP) data available in the SP-API...all kinds of things we can do with this. What should we build first? Todd VanderStelt Andy Thompson David G. Timurtek Bizel MixShift https://lnkd.in/gQpAYB8m (see SQP section)

68

24 Comments

I met one AI development agency founder, and they said our 50% RAG apps failed in production last year. Because teams don’t know how to handle: chunking + embeddings + vector DB + reranking + hallucinations + evaluation + monitoring. So I'm sharing a RAG Playbook that shows exactly how to build production-ready RAG systems, with practical examples. ✅ What you’ll learn from this playbook This playbook helps you go from: “RAG demo is working” → “RAG system is reliable in production” You’ll learn how to: - reduce hallucinations using prompting techniques - use advanced chunking methods (not random splits) - select the right embedding model for your use case - choose the best vector database (based on real needs) - add reranking to improve relevance - follow clear steps to build an enterprise-grade RAG system - run evaluation before production using key scenarios - monitor and optimize performance after deployment - track RAG quality using 4 powerful metrics If you’re building a RAG-based feature for your product, this playbook will save you weeks of trial and error. ✅ Save this post (you’ll need it later) 🔁 Repost for your team/network ➕ Follow Shubham Vora to learn more on AI agents.

139

77 Comments

Today Cohere is introducing our newest model, Embed 4! It brings enterprises state-of-the-art search and retrieval to securely power their AI agents and business applications. Unlike other embedding models, Embed 4 natively understands complex multimodal business materials across text and images – tables, graphs, code, diagrams, tech drawings, invoices, and more – to accurately and efficiently surface the most relevant insights for your work. Embed 4 brings enterprises: - Unmatched multimodality across PDF documents and presentation slides  - A context length that can handle documents up to 200 pages long (think product manuals, clinical trial reports, M&A due diligence files, and more) - Multilingual capabilities in 100+ languages  - Optimizations for regulated industries like finance, healthcare, and manufacturing - Secure deployment options to keep your data private We’re excited to provide enterprises secure, accurate, and efficient search and retrieval! https://lnkd.in/gJfhbFYs

270

11 Comments

Today the Pulse team published a deep dive on why a single “accuracy” score doesn’t tell you if a document extraction system will survive in production. The goal here is to lay out an introductory but still rigorous evaluation methodology - we have an exciting open-source benchmark building on this research coming out very soon. Let’s do the math: take 1,000 pages, each with 200 data elements. A model that’s 98% “accurate” on paper still produces 4,000 incorrect values. Now make some of those: 1/ Broken reading order that scrambles multi-column layouts 2/ Tables with shifted columns or missing headers 3/ Cross-page context lost entirely That’s enough to silently corrupt an entire dataset without throwing a single error. We’ve processed hundreds of millions of pages and built a multi-axis evaluation framework to measure what actually matters: reading order validation, region-level ANLS, reading order accuracy, TEDS for table structure, and continuity checks across page boundaries. The result? Fewer silent data corruptions, more predictable performance, and pipelines that keep working on the next million documents you haven’t seen yet. Full technical write up in the comments!

36

2 Comments

LLMs aren't the end—they’re just the beginning. Had an insightful discussion yesterday on the future of Advanced Machine Intelligence with Sudarshan, Paras and Pratyush. We explored why current large language models (LLMs), despite their impressive capabilities, fall short in critical areas: - Limited world modeling: Today's LLMs learn only discrete token-based representations, but genuine intelligence needs continuous world models (think intuitive physics from videos). - Persistent memory: We need models with large-scale associative memory, not merely token prediction. - Planning & Reasoning: True intelligence can plan, reason, and create novel solutions, something LLMs currently simulate but don't genuinely achieve. - Control & Safety: Rather than fine-tuning after training, control and safety must be intrinsic—built in by design. A surprising fact from our chat: 📌 A four-year-old child has already processed more sensory data than today's largest LLMs! There's still much to unlock. We're just scratching the surface. Thanks to Yann for these powerful insights, pushing us towards the next frontier of AI. More details on the discussion here: https://lu.ma/aeenojoc #FrontierAI #MachineLearning #SelfSupervisedLearning #AIResearch #ThoughtLeadership

233

11 Comments

LLMs are powerful, but they are stateless machines. They don’t carry memory or intent, they work with the context you provide at that moment. The same model can feel brilliant in one scenario and completely off in another. The difference is not the model, but the context and use case we design around it. The technology is already here. The real opportunity lies in how we shape it to solve real problems.

103

11 Comments