Ian Cook

Today we’re launching Inference Frontier: a builder-to-builder initiative focused on the inference craft Training breakthroughs get most of the attention. But once models hit production the real work begins That’s where teams run into the hard stuff: bursty traffic, KV cache pressure, batching vs. latency tradeoffs, routing, scheduling, cost / token at scale, and more We keep hearing the same thing from customers. Many teams are solving these problems in parallel, but the learnings rarely get shared Inference Frontier is designed to surface that work We’ll publish technical deep dives from teams running on top of inference systems, with a focus on architectures, optimizations, tradeoffs, and before-and-after improvements others can actually learn from. Submissions are now open if you or your team recently: improved latency or throughput reduced cost per token introduced a novel serving or scheduling approach solved a production inference problem others can learn from Super grateful to have an incredible judges helping evaluate submissions: Simon Mo Dylan Patel Olga Megorskaya George Cameron Song Han Braden Hancock Ryan Hanrui Wang Ujval Kapasi Laurelle Roseman Danila Shtan We’re also kicking things off with early deep dives from FlowGPT.ai Revolut monday.com If you’re building on inference systems (or know a team that is) nominate them https://lnkd.in/gb3G3Jfe

68

6 Comments

My Take on “Agentic” Frameworks and Real AI Engineering: There’s a lot of excitement around “agentic” frameworks, OpenAI’s AgentKit, LangChain, LlamaIndex, and others, all promising to simplify how we build intelligent systems. They’re great for quick prototypes, but when you’re designing production AI products that have to last, the picture changes. To me, an agent isn’t a visual node or a black-box workflow, it’s simply a modular, testable class that performs a defined task using an LLM or transformer model under strict control. In other words, it’s just good software design: clear inputs and outputs, deterministic orchestration, and measurable results. I prefer to build hybrid systems that combine structured engineering with state-of-the-art NLP and transformer architectures, using the LLM as a controlled component, not the entire system. That keeps the logic transparent, debuggable, and adaptable as models and requirements evolve. Frameworks like AgentKit can still help at the edges, for evaluation, UI, or quick orchestration, but the core intelligence and logic should always live in code you own and understand. That’s how you build AI that’s not only powerful today, but maintainable tomorrow. #AI #MachineLearning #LLM #SoftwareEngineering #NLP #ProductDesign

49

4 Comments

Finding the right mapping data or APIs shouldn't require a PhD in computer science. I've been compiling a table of data sources, from satellite data, to elevation data and even recently started on mapping and vector data. I was doing it for my own sanity, but wanted to solicit feedback. My next idea is to build a tool to help users determine which source is ideal for whatever their use case is, but I want to make sure the underlying data is solid first: - What am I missing? - Providers I should add? - Important attributes to track? - Common use cases where people struggle to find the right data? Let me know what you think, you can even open a ticket on GitHub here: https://lnkd.in/g8p96KJ8

53

19 Comments

> run it on prod db Sure, if you give an #AI agent write access to production, you're asking for trouble. But, Anthropic, if this is what Claude Code suggests at 2am in auto-mode after I ask it to write an ALTER TABLE, let's just say there's a problem on the tool side too 🙄

8

A week after Peter Steinberger joined OpenAI, one thing feels clear. Multi-agent systems are moving from experimentation to architecture. The conversation is shifting beyond standalone frontier models to coordination. How agents plan, delegate, communicate, and execute across systems, reliably and at scale. That orchestration layer will shape the next generation of AI products. It’s an important moment for teams building in agentic AI.

7

Session date: May 01, 2026, noon CST Math challenge: Can you explain Compton’s ring experiment? New prompting with Andrew Ng (repo) How to prepare your data for the future life with Snowflake or FreeEed (repo) AI News Beyond Vibe: more spec-driven development (repo) How to build with Anthropic Claude Console (course)

2

1 Comment

Introducing "Building with Llama 4." This short course is created with Meta, and taught by Amit Sangani, Director of Partner Engineering for Meta’s AI team. Meta’s new Llama 4 has added three new models and introduced a Mixture-of-Experts (MoE) architecture to its family of open-weight models, making them more efficient to serve. In this course, you’ll work with two of the three new models introduced in Llama 4. First is Maverick, a 400B parameter model, with 128 experts and 17B active parameters. Second is Scout, a 109B parameter model with 16 experts and 17B active parameters. Maverick and Scout support long context windows of up to a million tokens and 10M tokens, respectively. The latter is enough to support directly inputting even fairly large GitHub repos for analysis! In hands-on lessons, you’ll build apps using Llama 4’s new multimodal capabilities including reasoning across multiple images and image grounding, in which you can identify elements in images. You’ll also use the official Llama API, work with Llama 4’s long-context abilities, and learn about Llama’s newest open-source tools: its prompt optimization tool that automatically improves system prompts and synthetic data kit that generates high-quality datasets for fine-tuning. If you need an open model, Llama is a great option, and the Llama 4 family is an important part of any GenAI developer's toolkit. Through this course, you’ll learn to call Llama 4 via API, use its optimization tools, and build features that span text, images, and large context. Please sign up here: https://lnkd.in/gXKeipht

2,830

111 Comments

Most retrieval systems optimize for similarity. Very few optimize for certainty. When I started building Knolo, the goal wasn’t to compete with vector databases. It was to question an assumption: that retrieval must be probabilistic to be useful. Most teams accept non-determinism as the cost of working with language models. Rankings shift. Results drift. Evaluations become fuzzy. You can measure trends, but not guarantees. That’s fine for prototypes. It’s a problem for infrastructure. Knolo is built around a different constraint: the same input should produce the same retrieval artifact, every time. Portable. Inspectable. Stable across environments. Not because determinism is fashionable — but because it changes how you test, version, and reason about systems. Once retrieval becomes reproducible, it stops being magic and starts being engineering. And engineering is easier to scale than magic. https://www.knolo.dev/ #Engineering #SystemsThinking #Architecture #SoftwareDevelopment #Founders

3

🚨A strong signal from Databricks . Their new document parsing model reportedly outperforms frontier systems like GPT-5 and Claude — at up to 5× lower cost. Beyond the benchmark, what matters is what this unlocks: the ability to turn the world’s unstructured data (PDFs, tables, reports) into structured, machine-readable inputs for AI agents. This is the real bottleneck most organizations face — not model capability, but data accessibility. The infrastructure race is shifting from training models to connecting them to enterprise knowledge. Follow MindXO for weekly insights on AI strategy, infrastructure, and readiness shaping enterprises. #SignalOfTheDay #AITransformation #Databricks #EnterpriseAI #AIInfrastructure #DataStrategy #AgenticAI #AIReadiness #MindXO #AIEconomy #GCCInnovation #DigitalTransformation #ArtificialIntelligence #AIstrategy

1

So the new Gartner MQ for CDPs is out. I sometimes wonder if (like that annoying classmate) Gartner deliberately provokes for attention, but in any case, buyers should not rely on their rankings. Rather than a long post, I'll offer a short list of observations: - As usual, G misses big chunks of the market - Note some quite radical shifts versus 2024; no: the market actually didn't move that fast....G is just always five years out of date, and so any given quadrant can't be trusted - "Ability to execute" (Y axis) always correlates with vendor sales and marketing spend - "Vision" (X axis) is PPT theater and fundamental mis-reading of what enterprises look for in this market - Never trust software that Salesforce builds itself... 😉 Link to last year's post on this topic in the first comment....

124

23 Comments

Reactor Data's knowledge base is live! 🎁 This is our open treasure trove of hands on tutorials and best practices for creating more AI- and business-relevant data faster at lower cloud and engineering expense (with help from 🤖 Electron, of course). Check it out: https://lnkd.in/ghhVHp-N

68

4 Comments

New: Databricks is launching a startup accelerator for seed and pre-seed companies, offering up to $250,000 to startups in the form of funding and credits https://lnkd.in/eHjG-9T8

33

3 Comments

As spatial data grows massive, ad-hoc scripts just don’t cut it anymore. We need robust orchestration to handle the scale, and that is where Apache Airflow is quickly becoming non-negotiable for geospatial engineers. But orchestration needs a scalable compute engine to back it up. That’s why I’m teaming up with Tamara Janina Fingerlin from Astronomer for a hands-on workshop next Tuesday and Thursday. We are going to show you exactly how to combine the orchestration power of Airflow with the spatial compute power of Apache Sedona and Wherobots. We’ll be building a robust pipeline live taking raw data from NOAA and Overture Maps, running spatial joins on buildings/neighborhoods, and outputting a dynamic visualization. What we’re covering: - Modern orchestration (Astro IDE, Debbie Gormal, Kathy Alteen-Reid) - Advanced features (Dynamic Task Mapping, "Human in the loop") - Scalable spatial processing with WherobotsDB 🗓️ When: Tuesday, Feb 10, 2026 | 9am PST / 12pm ET Bonus: Submit your pipeline to get an official certificate. Join us to see what a modern spatial data stack looks like in action. Registration in the comments.

99

3 Comments

This week, Claude’s coworker plugins (legal, product, research, etc.) shook SaaS stocks — with some calling it a “SaaSocalypse.” What’s really happening is simpler. They’re codified expertise — structured prompt libraries + agents delivering professional-grade work, without heavy SaaS UI. SaaS isn’t dead. But defensibility is shifting — toward domain depth, proprietary data, and distribution. #AI #SaaS #ClaudeCode #ClaudeCowork #SaaSocalypse

22

So many reactions to OpenAI’s new “agent builder.” While the excitement is palpable, it's important to recognize that orchestration isn't a new concept. Frameworks like LangChain, LangGraph, CrewAI, and visual builders such as Langflow, Make, Zapier, and Flowise have been paving the way for years, making orchestration possible. OpenAI's new tools have certainly drawn fresh attention to this critical layer of infrastructure, but we must also acknowledge the often-overlooked eval layer that is equally important. This distinction matters. Every major platform moment follows a similar trajectory: builders recognize the potential first, and the market catches up later. However, once orchestration becomes accessible to everyone, it transitions from being a differentiator to a necessary infrastructure. The challenge now lies not in connecting steps but in connecting meaning—understanding what to connect it to, the business domain, the data, and how to leverage orchestration effectively. This shift is central to my exploration in The AI Executive’s Handbook: “Prompt engineering was a parlor trick. Context engineering is an architecture.” The orchestration layer connects tools. The context layer connects purpose. The domain layer connects value. This is where the next competitive advantage resides—not in the quantity of agents built, but in their understanding of the world they operate in. Stay tuned for a request for help in selecting a cover tomorrow. #AI #ContextEngineering #AgenticAI #TheAIExecutivesHandbook #Governance #LangChain #Strategy #OpenAI

27

4 Comments

A pattern I keep seeing in platform decisions: Teams optimize for *technical capability* before asking about *decision frequency*. Streaming pipelines. Always-on compute. Sub-second freshness. But the actual decisions happen: • once a day • once a week • sometimes once a month We learned this lesson painfully during migrations: If no one acts on data in minutes, real-time becomes an expensive habit. Start with how fast decisions are made. Earn your way to complexity. What’s one “real-time” system you’re running that nobody checks in real time?

13

lya Sutskever has confirmed that LLMs are not scaling with massive gains anymore Enough if Compute continues to scale, but data is not keeping up. New or synthetic data doesnt significantly improved performance. In next wave models can actually get smaller and do one job right. 1)Agentic AI system is the future 2)LLMs can be replaced with SLMs to create optimal systems 3)LLM/SLM not being absolutely deterministic in outputs wont matter as long as system design is robust.

46

3 Comments