Eddie G.

The GPU budget isn't what's killing your AI ROI. It's the silent compute burn happening before the model even touches the data. 🛑 Everyone is obsessing over the cost of model training, but here is the unpopular engineering truth: for most enterprise AI projects, data preprocessing consumes significantly more compute than actual model training. Heavy workloads like deduplication, normalization, and label cleaning are massive cost centers. Yet, while engineering teams meticulously track GPU hours, almost nobody instruments their data pipelines. You cannot optimize what you do not measure. If you lack granular visibility into your preprocessing compute, you are flying completely blind on a cost center that can easily exceed your entire training budget. Stop bleeding compute in the dark. Instrument your data pipeline. Take control of your AI unit economics today at https://zurl.co/bieZo. 👇 #DataEngineering #FinOps #EnterpriseAI

6

AI Tech Stack Funding this week: $5.6bn Eleven companies that raised this week in the AI infrastructure / tool stack space: CoreWeave – Post-IPO Equity – $2bn: A cloud-based AI infrastructure company offering GPU cloud services to support large-scale AI and machine learning workloads. StepFun – Series B – $718.1m: An artificial intelligence company focused on advancing toward artificial general intelligence (AGI). AstraSync AI – Seed – Undisclosed: Know Your Agent platform providing identity, trust, and verification for the agentic economy. Databricks – Debt Financing – $1.8bn: A data and AI platform unifying data engineering, analytics, and machine learning on a lakehouse architecture. LiveKit – Series C – $100m: A cloud platform enabling developers to build, deploy, and manage real-time communication and AI-driven applications. AppliedAI – Series B – Undisclosed: Provides AI-augmented automation for mission-critical business workflows. Upscale AI – Series A – $200m: Redefines networking with purpose-built solutions for high-performance compute environments. Baseten – Venture – $300m: An AI infrastructure platform integrating machine learning into production business systems and workflows. humans& – Seed – $480m: A frontier AI research and product lab focused on human-centred AI systems and interaction models. Thunder Compute – Seed – $4.5m: One-click GPU instances designed to deliver compute at significantly lower cost. NENNA.AI – Seed – Undisclosed: An AI enablement platform focused on the secure application of AI technologies. #AI #ArtificialIntelligence #AITechStack #AIFunding #AIEcosystem #AIInfrastructure #MachineLearning #ML #Startups #TechFunding #VentureCapital

6

Lovable hit $100 million ARR 8 months from launch. Anysphere's Cursor took 12 months to hit $100 million Before that, It took Wiz 18 months to reach $100M And Deel 20 months Ramp 24 months All these companies exhibited #escape #velocity. If you look at how they conducted business, these companies focused on #distribution early on in the company's journey. "The game of #startups is if the startup can gain #distribution faster than the incumbent gain #innovation ", as Alex Rampell wrote in his 2015 blog post Alex Rampell in his blog post “Distribution vs Innovation” As Lovable's stats demonstrate, the escape velocity is becoming larger and more aggressive, particularly with #AI. Will we see companies reaching $100M ARR in 6 months? Will we see solopreneur companies achieving the same escape velocity? The question for other #startups, those that funded but not achieving quite the same escape velocity -- what are you doing to steer yourself onto the escape velocity path? As Brian Balfour wrote in this excellent blog: "The Big Squeeze: Why Escape Velocity Is More Important Than Ever"-- #Distribution is the seed that unlocks harvest (I recommend every startup founder read this blog, link in the comments) - Who can you integrate with? - Which channels can you attain distribution faster than others? - Who should you partner with to give you that market #uplift? #distribution is indeed more important than ever. If you are a startup and you are not asking these questions from day 1, you will be left behind.

62

13 Comments

Building trust in LLMs starts with rigorous evaluation, not intuition. Great breakdown on why structured, evidence-based evals are becoming essential for production-grade models.

14

3 Comments

1️⃣7️⃣9️⃣ - Backprop by hand through the output layer Derived output layer gradients and traced loss back through cross entropy, following along with Andrej. 20 min

1

My notes from the AI Powered Engineering Productivity talk at Snowflake Toronto 👇 📈 The Real Productivity Gains (Not What You Think) 50% increase in code shipped per engineer. But here's what's actually driving it: → AI writes documentation (and keeps it current) → AI generates better test coverage → AI debugs production logs Result: More stable systems, less firefighting. 2️⃣ Two Use Cases Worth Using: 1. LLM integrated into Slack for support questions 2. LLM pointed at production logs for debugging Both reduced time teams spend on repetitive work. 📊 The Only Metric That Matters: Forget story points. Rather, measure time to productize. How fast can you go from idea → customer using it? That's it. 🤝 How to Interview Engineers in 2025 Stop asking leetcode puzzles. Give them AI tools, ask them to solve a real problem and have them explain their solution. You'll learn how they think, not what they memorized. 🛠️ On "Will AI Replace Engineers?" Steven Woods (Snowflake CTO) nailed it: "We don't hire engineers to write code. We hire intelligent problem solvers who understand the business and happen to be technical and relentless. Those people aren't replaceable." If you're just coding, you're at risk. If you're a problem solver, you're not. Overall a really great talk. Thanks to Qaiser Habib and the Snowflake team for putting it all together. Thanks to the speakers: John Abd-El-Malek, Josh Zana And amazing panel: Mohit Gupta, Leho Nigul and Steven G. Woods P.S. Feels good to be back in the arena. See you around!

41

3 Comments

xAI is sharing spare GPU capacity with Anthropic. Their cluster utilization was reportedly running at 11%. The industry average isn't much better — 40%. This isn't an xAI problem. It's a structural one. Most clusters are reserved by a single tenant. But compute demand from any one team is inherently uneven: quiet during planning cycles, maxed out before a model deadline. The gap between those two states gets eaten by idle GPUs no one is using, while teams without reservations queue for hours to get a single node. Velda is making compute fluid. If you reserve GPUs with us, your unused capacity doesn't go to waste: it converts automatically into spot credits you can draw on later. You will have prioritized access to the spot pool, which comes from unused compute of reservations that would be otherwise wasted. Velda allows you to run GPU jobs with a simple command prefix; Preempted jobs retry automatically from last check-point. Getting cluster access should feel like using your workstation, no manifest, no extra setup. And we're not going to mark up the hardware. You pay what the GPU provider charges, no platform tax on top. If you have an existing provider relationship, we'll work with them directly. The capacity is there. It just needs to move.

23

6 Comments

𝗕𝗥𝗘𝗔𝗞𝗜𝗡𝗚: OpenAI just embraced open source with gpt-oss-120b for HPC and gpt-oss-20b for consumer machines. It's great to see this type of response from OpenAI. I speak with many F500 AI leaders and they are extremely wary of: - Relying on single closed-source vendors with third-party risk - The risks of not having data portability from day one - Privacy and regulation that will inevitably catch up The reality with closed models is that even if your data stays in your databases, it's getting passed through an exchange loop where the vendor can see, access, and understand everything before passing it back to you. Enabling enterprises to run local models returns control to users and enterprise AI teams. It's a really good sign toward a multi-model, multi-context world.

26

1 Comment

2–4x faster LLM output. Same quality. Lower cost. Meet Speculative Decoding 🤯 Traditional text generation—where Large Language Models (LLMs) predict one token at a time—is a major bottleneck for real-time performance. But there’s a smart solution that can drastically speed things up without sacrificing output quality: Speculative Decoding. Think of it as “draft and verify” for language generation: 🔹 A smaller, faster draft model quickly speculates what the next few tokens might be. 🔹 Meanwhile, your larger target model checks these guesses in parallel, verifying multiple tokens in a single forward pass. This parallelism is the game-changer. Instead of waiting for one token at a time, the system can validate several in one shot—massively accelerating output. To keep quality high, a clever step called rejection sampling compares token probabilities from both models: If the big model agrees (or is more confident), the draft token is accepted ✅ If not, the system rolls back and lets the large model generate the correct token 🔁 💡 The result? Same high-quality output, generated up to 2–4x faster. Why it matters: ✔️ Speed: Drastically faster inference—real-time responses are now within reach ✔️ Efficiency: Better GPU utilization by offloading to smaller models ✔️ Cost Savings: Fewer compute cycles, lower bills ✔️ Quality: No compromise—final output matches the large model’s gold standard Speculative Decoding is a practical, production-ready optimization. If you’re deploying LLMs at scale—or plan to—it’s a technique worth exploring. 🔍 Are you already using it? What’s your biggest challenge in LLM inference today? #LLM #AI #SpeculativeDecoding #MachineLearning #NLP #GenerativeAI #InferenceOptimization #DeepLearning #TechInnovation #ArtificialIntelligence

1

You don't need a separate reference model for RL. We built an RL pipeline built on a multi-turn-focused modification of Self-Distillation Policy Optimization (SDPO) that uses the same model in two roles: student and teacher. The difference? Context window. → The student generates actions with no knowledge of the outcome. The teacher re-evaluates the same tokens with hindsight: the observation from each action, terminal test output, and a successful rollout from the same prompt. The per-token advantage is just the logit gap between these two views. → This gives you dense, per-token credit assignment instead of a single scalar reward for the whole trajectory. Think tokens get 0.25× weight (loosely guided, not tightly constrained). Action tokens get full gradient. System/user/tool result tokens get nothing. → The "teacher" is not a frozen checkpoint or a separate reward model. It's your current policy, conditioned on richer information. This is well-defined under MoE because same parameters with different routing is normal operating behavior. → We use a tiered reward system with hard gates only for truly unrecoverable failures: sandbox escapes, test corruption, harness crashes, timeouts, infinite loops. Everything else like wrong directory, failed commands, incorrect approaches are learnable signal, not wasted compute. The key insight: most of the information you need for credit assignment is already inside the model. You just need to ask it the right question with the right context. On LiveCodeBench v6 SDPO reached 48.8% vs GRPO's 41.2% beating Claude Sonnet 4 with an 8B model! That's bonkers.

5

The New Interface Isn’t Visual — It’s Conversational On July 17, OpenAI didn’t just ship another feature. They dropped a signal: ChatGPT Agents. Not GPT-5 (yet), but arguably just as disruptive. At first? Clunky. Slow. Skeptics were loud. But step back, and you’ll see the bigger play: 👉 Replacing GUIs with AI-driven APIs. 👉 Turning agents into the new entry point to the internet. ChatGPT now handles 2.5 billion prompts/day — that’s already ~18% of Google’s annual search volume. And this is just the beginning. But it’s not just about language anymore. The real differentiator? → Agent architecture: sandboxing, execution, tools, planning, feedback loops. I outlined the current technical approaches powering general-purpose AI agents today. Full Details here: 🔗 https://lnkd.in/dCpUHBFe Missed something? Call it out 👇

5

2 Comments

🚀 Stop treating AI as just a "faster autocomplete." Welcome to the era of **Agent-Native Engineering**. A new operating model is completely restructuring how software teams work. Instead of just helping you type, AI agents are now acting as actual contributors—taking on scoped tasks, opening pull requests, and iterating on feedback asynchronously. Here is what the new engineering workflow looks like: 🔹 **Simple tasks** ➡️ One-shotted by agents from a clear prompt. 🔹 **Manageable tasks** ➡️ Delegated to background agents with built-in review cycles. 🔹 **Complex tasks** ➡️ Kept with engineers using synchronous coding agents. The result? Engineers spend less time writing boilerplate code and more time doing what matters most: scoping problems, reviewing work, and shaping the product. 💡 Check out this great read from Cofounder on the future of software development: https://lnkd.in/dJX5phfz #SoftwareEngineering #ArtificialIntelligence #AIAgents #FutureOfWork #AgentNative #TechTrends #SoftwareDevelopment

4

Just getting started with LLMs? Don’t let the jargon slow you down. This 5-part LLM Glossary is your shortcut to mastering the fundamentals👇 1️⃣ Model Types • Foundation Model: Pretrained on large-scale, diverse datasets to learn general capabilities. • Instruction-Tuned Model: Further trained to follow user instructions precisely. • Multi-modal Model: Handles multiple input/output formats like text, images, and audio. • Reasoning Model: Optimized for logic, problem-solving, and step-by-step thinking. • Small Language Model (SLM): Lightweight models for fast, efficient, on-device tasks. 2️⃣ Training & Fine-Tuning • Pretraining: Initial learning phase using massive, diverse data sources. • RLHF (Reinforcement Learning with Human Feedback): Aligns model behavior with human preferences. • DPO (Direct Preference Optimization): Uses ranked preferences instead of rewards for training. • Synthetic Data: AI-generated data used to supplement real datasets. • Fine-Tuning: Targeted retraining on specific domains or tasks. • LoRA / QLoRA: Techniques for efficient low-resource fine-tuning. • Guardrails: Rules or filters applied to enforce safety, ethics, and compliance. 3️⃣ Prompt Engineering • System/User Prompts: Define model behavior and user context in a session. • Chain-of-Thought (CoT): Prompts that guide the model to reason step-by-step. • Few-Shot / Zero-Shot Learning: Demonstrating tasks with few or no examples. • Prompt Tuning: Optimizing prompts via training for specific outcomes. • Context Window: The total number of tokens the model can "remember" per interaction. 4️⃣ Inference & Generation • Temperature: Adjusts output randomness; lower values are more deterministic. • Max Tokens: Limits the number of tokens in a generated response. • Seed: Controls reproducibility of outputs. • Latency: Time taken by the model to respond. • Hallucination: When the model generates factually incorrect but plausible-sounding information. 5️⃣ Retrieval-Augmented Generation (RAG) • Retrieval: Fetching relevant data from external sources before generation. • Semantic Search: Search based on meaning, not just keywords. • Embeddings: Vector representations of text used for similarity matching. • Chunks: Dividing documents into smaller segments for better retrieval. • Vector Databases (VectorDBs): Stores embeddings for fast and accurate search. • Reranking: Rescoring retrieved documents to prioritize relevance. • Indexing: Structuring data for efficient retrieval during generation. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

232

36 Comments

This week, Claude’s coworker plugins (legal, product, research, etc.) shook SaaS stocks — with some calling it a “SaaSocalypse.” What’s really happening is simpler. They’re codified expertise — structured prompt libraries + agents delivering professional-grade work, without heavy SaaS UI. SaaS isn’t dead. But defensibility is shifting — toward domain depth, proprietary data, and distribution. #AI #SaaS #ClaudeCode #ClaudeCowork #SaaSocalypse

22

Introducing "Building with Llama 4." This short course is created with Meta, and taught by Amit Sangani, Director of Partner Engineering for Meta’s AI team. Meta’s new Llama 4 has added three new models and introduced a Mixture-of-Experts (MoE) architecture to its family of open-weight models, making them more efficient to serve. In this course, you’ll work with two of the three new models introduced in Llama 4. First is Maverick, a 400B parameter model, with 128 experts and 17B active parameters. Second is Scout, a 109B parameter model with 16 experts and 17B active parameters. Maverick and Scout support long context windows of up to a million tokens and 10M tokens, respectively. The latter is enough to support directly inputting even fairly large GitHub repos for analysis! In hands-on lessons, you’ll build apps using Llama 4’s new multimodal capabilities including reasoning across multiple images and image grounding, in which you can identify elements in images. You’ll also use the official Llama API, work with Llama 4’s long-context abilities, and learn about Llama’s newest open-source tools: its prompt optimization tool that automatically improves system prompts and synthetic data kit that generates high-quality datasets for fine-tuning. If you need an open model, Llama is a great option, and the Llama 4 family is an important part of any GenAI developer's toolkit. Through this course, you’ll learn to call Llama 4 via API, use its optimization tools, and build features that span text, images, and large context. Please sign up here: https://lnkd.in/gXKeipht

2,830

111 Comments

The hits keep coming at VSC 💡 Inception Labs – Stanford professor+team raises $50M and builds a 10x faster and 10x cheaper AI coding model with the performance of Gemini Flash / Haiku. AI veterans Andrew Ng and Andrej Karpathy are among the investors, alongside Nvidia, Menlo, Microsoft, Mayfield, Databricks, and Snowflake. 🧠 Sema4.ai – launches its enterprise AI platform designed to streamline complex data and document workflows 🔒 Confident Security – debuts OpenPCC, an open-source standard that encrypts data fed into AI tools 📈 Noetica – offers perspective on tightening private credit terms and market risk 💊 Foresite Capital –Biotech 2050 podcast to discuss AI and investment strategy. ☀️ Sesame Solar – powers long-endurance drones and off-grid operations with its solar hydrogen nanogrid cc: Stefano Ermon, Daniel Wertman, Paul Codding, Ram Venkatesh, Jonathan Mortensen, PhD, Jim Tananbaum, Lauren Flanagan

51

1 Comment

𝗔𝗿𝗲 𝗠𝗼𝗱𝗲𝗹 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝘁𝗼𝗰𝗼𝗹 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗳𝗿𝗼𝗺 𝗔𝗣𝗜𝘀? You may think APIs were enough for AI agents, Then when you looked into 𝗠𝗼𝗱𝗲𝗹 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝘁𝗼𝗰𝗼𝗹 (𝗠𝗖𝗣), you realize APIs are powerful, but they weren’t built for how LLMs operate. • 𝘼𝙋𝙄𝙨 𝙖𝙧𝙚 𝙜𝙚𝙣𝙚𝙧𝙖𝙡-𝙥𝙪𝙧𝙥𝙤𝙨𝙚 great for structured system interactions, but they rely on pre-defined endpoints, and agents need custom adapters to use them. • 𝙈𝘾𝙋 𝙞𝙨 𝙥𝙪𝙧𝙥𝙤𝙨𝙚-𝙗𝙪𝙞𝙡𝙩 AI agents can ask what tools or data are available at runtime, and everything is exposed in a consistent, machine-readable format. One interface, Multiple services, No redeploys. Many MCP servers are just wrapping traditional APIs - but with a layer that actually speaks agent. Over the past few months, a range of protocols have emerged to standardize how AI agents communicate, collaborate, and function. As systems grow more autonomous, having defined protocols is no longer optional — it's essential. This 10 of the most prominent AI agent protocols shaping the future of intelligent systems: ➤ ACP (IBM) – Defines standard interfaces for agent interaction and lifecycle management. ➤ AGP (Industry) – Enables message transformation and access control between agents and external systems. ➤ A2A (Google) – Powers structured communication in multi-agent environments like Gemini and Project Astra. ➤ MCP (Anthropic) – A unified model context protocol for memory injection and tool usage in LLMs. ➤ TAP (LangChain) – A standardized JSON schema for tool abstraction and execution. ➤ OAP (Community) – An open protocol for framework interoperability across agent runtimes. ➤ RDF-Agent (Semantic Web) – Leverages SPARQL and linked data for semantic agent communication. ➤ AgentOS (Proprietary) – Provides a runtime stack for long-lived, enterprise-grade agents. ➤ TDF (Stanford) – Task schema for modular prompt planning and goal coordination. ➤ FCP (OpenAI) – A standardized way to invoke LLM functions with schema enforcement. Each protocol addresses a unique layer in the agentic AI stack — from communication to memory, task coordination to function invocation. 𝗜𝘁’𝘀 𝗻𝗼𝘁 𝗔𝗣𝗜𝘀 𝘃𝘀 𝗠𝗖𝗣 - 𝗶𝘁’𝘀 𝗮𝗯𝗼𝘂𝘁 𝗺𝗮𝗸𝗶𝗻𝗴 𝗔𝗣𝗜𝘀 𝘂𝘀𝗮𝗯𝗹𝗲 𝗮𝘁 𝗔𝗜 𝘀𝗰𝗮𝗹𝗲. #LLM #MCP #APIs #AIagents

3

#AI coding #agents are changing how I evaluate vendors. A strong argument for products is if they can be configured purely through code and #API / #MCP. Not because they have their own AI features - but because they let your AI agent do the work. Lightdash is a great example. It is not the most mature BI tool, but probably one of the most AI-ready - because they made all data and configuration accessible via API and gave AI agents the context to act on it, so Claude Code can end-to-end create dashboards by modelling data in the data warehouse and then build the visualizations. If your product is not API-first, AI cannot bring in outside context or act autonomously within it. That is becoming a real competitive disadvantage. https://lnkd.in/e6Gp3MGj

53

14 Comments