Laurence Moroney

New course to bring you up to state-of-the-art at using AI to help you code: Build Apps with Windsurf's AI Coding Agents, built in partnership with Windsurf and taught by Anshul Ramachandran! AI-assisted IDEs (Integrated Development Environments) make developers’ workflows faster, more efficient, and much more fun. Agentic tools like Windsurf are more than just code autocomplete—they are collaborative coding agents that help you break down complex applications, iterate efficiently, and generate code that spans multiple files. Although a lot of coding assistants share the same underlying large language models for planning and reasoning, a major point of distinction is how they handle tools, keep track of context, and stay aligned with your intent as a developer. For instance, if you make modifications to a class definition in your code and make the same modifications to other classes in the same directory, you might tell the AI agent "Do the same thing in similar places in this directory." Here, tracking your intent means understanding that “the same thing" refers to that recent edit you just made, which must be followed by appropriate search and tool-calling to implement the changes. In this course, you'll learn the inner workings of coding agents, their strengths and limitations, and how to use Windsurf to quickly build several applications. In detail, you'll: - Build a mental model of how agents work by combining human-action tracking, tool integration, and context awareness to carry out an agentic coding workflow. - Learn the challenges of code search and discovery and how a multi-step retrieval approach helps coding agents address them. - Use Windsurf to analyze and understand a large, old codebase and update it to the latest versions of the frameworks and packages it uses. - Build a Wikipedia data analysis app that retrieves, parses, and analyzes word frequencies. - Enhance the performance of your Wikipedia analysis app by adding caching, and through this, also learn how to course-correct when the AI agent produces unexpected results. - Learn tips and tricks such as keyboard shortcuts, autocomplete, and @ mentions to quickly call on agentic capabilities. - Use image/multimodal capabilities of the AI agent to increase your development velocity; you'll see an example of uploading a mockup with sketched-out UI features, and ask the agent to use that to build new functionality to an app. By the end of this course, you’ll understand agentic coding in-depth and know how to use it to make your development process much faster, more efficient, and enjoyable. Please sign up here! https://lnkd.in/gaw3vq_u

2,293

106 Comments

The Generative AI era is fading… at least according to Nvidia CEO Jensen Huang, who took the stage this morning at @VivaTech in Paris.   What’s next? Agentic AI, which builds on everything we’ve learned so far.   Inside the stiflingly hot Dôme de Paris, 4,000 attendees hung on Huang’s every word as he mapped out the future of AI. Powered, of course, by Nvidia.   Alongside his Agentic AI vision (featuring a Perplexity-based example), Huang leaned into his usual pitch around digital twins. But he went further, expanding his robotics vision to include all forms of mobility, predicting that “physical AI” is just around the corner. Physical AI includes anything that moves, like cars, boats, airplanes and, yes, robots.   He showed a demo of Nvidia’s autonomous driving software being tested in the wild, which showcased, interestingly, parts of my hometown just south of San Francisco.   The most impressive reveal for me? Nvidia’s new line of small AI computers: a mini-PC, a notebook, and a full-sized “server” PC, all designed to mimic the company’s Grace Blackwell rack-mounted systems at a much smaller scale. Needless to say, I want one in my house.   Jensen compared the new system to the original GPU that launched Nvidia more than 30 years ago and explained how today's machines might serve as QPUs (quantum processing units), acting as co-processors to accelerate quantum computing.   He also shared a jaw-dropping stat: the interconnect on Nvidia’s $3M+ Grace Blackwell servers moves 130 terabytes per second. That's more than the peak traffic of the world’s entire internet.   Europe got a major spotlight, too, with Huang showcasing AI factories, robotics projects, and partners across the continent and UK. A 10x boost in localized AI power is coming by the end of next year, he promised, as Nvidia made a strong bid to win over the European technorati.   It was the company’s first European GTC, and with this keynote they planted a long-term flag.   Best quote of the day: “It turns out the world is beautiful—and math is beautiful too.”   Compared to the 2024 GTC keynote in San Jose, this one was notably less polished. A few reveals didn't actually reveal on time - and at one point Jensen struggled to advance the slides.   “This is what happens when you only rehearse once,” he quipped.   Still, the crowd was with him, and the keynote was warmly received.   And then he brought down the house with a presumably autonomous robot from Disney’s Imagineering division.   Are humanoid robots and quantum computing really just around the corner? Are you ready?   Let me know in the comments.   #VivaTech #NvidiaGTC #AI #Robots #VivaTechAmbassadors  

431

21 Comments

🚀 Exciting Collaboration Announcement at CES: NVIDIA × Docling 🚀 We’re thrilled to share that NVIDIA and Docling are collaborating to deliver best-in-class performance for high-quality document conversion. This collaboration has already led to significant optimizations in Docling, leveraging batch processing to achieve up to a 4× speed-up compared to CPU-only workloads. ️Thanks to the close partnership with the NVIDIA RTX AI team, we were able to develop, test, and benchmark across multiple NVIDIA systems, including the powerful RTX 5090—pushing the limits of document conversion performance on RTX PCs. Learn more about the NVIDIA RTX optimizations: https://lnkd.in/dpsHq7af Get started with Docling on RTX: https://lnkd.in/dsN4KHwt We’re excited about what’s ahead and look forward to enabling faster, higher-quality document processing for the community. Stay tuned for more Docling news! Many thanks to our awesome collaborators at Nvidia: Aslı Sabancı Demiröz, Douglas O'Flaherty, Sean Sodha and many more! cc: Abdel Labbi, Sriram Raghavan, Shankar Ramaswamy, Ed Anuff, Milan (Marquez) Higginbotham, Alejandro Cantarero #CES #IBM #NVIDIA #Docling #RTXAI #OpenSource #DocumentAI #Performance #AIInfrastructure

1,057

27 Comments

I'm happy to share that we've released Instella-Math: a reasoning-focused language model trained with long CoT reinforcement learning on AMD GPUs. It is fully open – its base model (Instella-3B-Instruct), reasoning SFT, and RL training are all fully open. In addition to commonly used math and science reasoning benchmarks, we also evaluate on a new benchmark, TTT-Bench, targeting strategic, spatial, and logical reasoning. Our model performs surprisingly well on TTT-Bench without any exposure to similar-style data during any stage of training. #AMD #LLM #ML #AI #HW #MI300X https://lnkd.in/gnkFp7uQ

206

11 Comments

Reinforcement Fine-Tuning (RFT) is a technique that adapts Large Language Models (LLMs) to intricate reasoning tasks. Dive deep into a comprehensive exploration of RFT with GRPO in this course. Learn how to construct reward functions crucial for steering LLMs towards enhanced performance on complex, multi-step reasoning tasks during the GRPO training process.

31

2 Comments

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models:  - Nemotron Speech ASR  - Nemotron 3 Nano 30GB in a 4-bit quant (released in December)  - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat Cloud, or run locally on an NVIDIA DGX Spark. Here's a write-up about the voice agent in the video above, the three NVIDIA models, how to deploy to production, and some fun optimizations if you're running locally on a single GPU: https://lnkd.in/gZWHR9GP Code is all here: https://lnkd.in/gvmJXjjx You can deploy these models to @modal cloud in a few minutes. (I love the Modal developer experience.) To run locally, you'll need to build a Docker container (because, you know, bleeding edge vLLM, llama.cpp, CUDA for Blackwell, etc). But the Dockerfile in the repo should "just work" on DGX Spark and RTX 5090. If you have trouble, or make patches to extend to other platforms, please let me know! Shoutout to Ben Shababo for getting all this running on Modal and doing really cool benchmarks work!

733

56 Comments

NVIDIA at #DISTRIBUTECH2025, with Ahsan Yousufzai ... ✔️ 𝗢𝗻 𝘁𝗵𝗲 𝗢𝗽𝗲𝗻 𝗣𝗼𝘄𝗲𝗿 𝗔𝗜 𝗖𝗼𝗻𝘀𝗼𝗿𝘁𝗶𝘂𝗺 - "It has to be a domain-specific model ... and this is what we are trying to do working with EPRI, to build this open source model, for the community, which they can use and really take it to the operations. That is a key thing, operations, right? If we can help the operations with #generativeAI agents, the guy who's making a decision, if we can help him with the recommendation, suggestion, you are saving millions of dollars for the industry." ✔️ 𝗢𝗻 𝘁𝗵𝗲 𝗼𝘃𝗲𝗿𝗮𝗹𝗹 𝗿𝗼𝗹𝗲 𝗼𝗳 𝗔𝗜 : " as DER's, and datacenters, all these industrial heavy loads coming in ... the transmission grid, the distribution grid ... everybody is trying ... they're under a lot of pressure. And, they see AI as one of the way they can solve those problems. While AI is creating some of those workloads, AI is the way you can solve those problems." ✔️ 𝗧𝗵𝗲 𝗿𝗼𝗹𝗲 𝗼𝗳 𝗔𝗜 𝗮𝘁 𝘁𝗵𝗲 𝗲𝗱𝗴𝗲 : "If we can make them intelligent and they can talk to each other, the amount of efficiency eventually leading towards autonomous grid, that's the goal ... It's a journey. It's a long journey ... But we have to bring intelligence to the far edge ... And that's where if you see a lot of these, we say 'painting the floor green' ... we have 47 plus partners, customers on the floor, taking those Jetsons, all the NVIDIA stack, and really enabling the grid, from a generation side to the transmission side, substation, down to the medium voltage, low voltage ... right across the board." I met up with Ahsan, Global Director Energy Surface at #NVIDIA, during DTECH a few weeks back in Dallas to get his takeaways from the event... Here are a few clips with a link to our full chat in a comment below. cc Irina Shekhovtsova, NVIDIA AI, NVIDIA Omniverse, NVIDIA Data Center #DTECH25 #Energy, #EnergyIndustry #GenAI, #AIFactory, #Technology, #FutureofEnergy For transparency, this was not a paid engagement, just me being me.

74

7 Comments

The future of AI isn't about more FLOPS. It's about memory. 🧠 David A Patterson (Turing Award winner, co-creator of RISC) & Xiaoyu Ma just dropped a paper that highlights the changing tide of AI. It's titled "Challenges and Research Directions for Large Language Model Inference Hardware" (link in comments). Current accelerators are training chips repurposed for inference - optimizing for the wrong metric entirely. What doesn't matter as much as it once did: Peak FLOPS. The new bottlenecks that matter: ➡️ Memory bandwidth ➡️ Memory capacity ➡️ Interconnect latency The other important points from the paper as a quick summary - these matter a lot as they help determine the future of AI inference, and which stock you are betting on 🤑⬇️ ❌ Inference ≠ Training - The autoregressive Decode phase is fundamentally memory-bound, not compute-bound. Current GPUs/TPUs are repurposed training chips that prioritize FLOPS - the wrong metric for inference. 🧠 Memory is the new bottleneck - Memory bandwidth, capacity, and interconnect latency are the decisive factors, not raw compute. The "Memory Wall" is real and accelerating. ⏩ Interconnect latency kills - For decode workloads, latency (not bandwidth) dominates. Small-message, high-frequency communication patterns across multi-chip systems are the choke point. 💨 Current hardware is mismatched - HBM costs keep rising while DRAM scaling slows. Accelerator architectures are built for training-style parallelism, not inference's sequential token generation of today. ✅ New AI trends amplify the problem - MoE, reasoning models, multimodal, long-context, and RAG all increase memory pressure and inter-chip communication - making inference even more memory-bound. 🔌 High Bandwidth Flash (HBF) - Proposed 10x memory capacity increase via stacked flash with HBM-like bandwidth. Perfect for read-mostly weights and slow-changing context. 🥞 3D stacking is the future - Vertical memory-logic integration delivers bandwidth gains with power efficiency. Thermal management and interface standardization remain challenges. 🚀 Latency-first network design - New topologies (tree, dragonfly, high-D tori) and in-network processing for collectives. Place compute near NICs, not in distant DRAM. 📈 New metrics required - We have to stop chasing FLOPS and instead measure performance per dollar, per watt (and ideally per CO2e). The industry needs a roofline model for memory-centric inference. #llm #genai #tokenomics #future #artificialintelligence #scaling #cloud #hardware #nvidia #amd #groq

169

21 Comments

We are excited to share that the new OpenAI models -- gpt-oss-20b and gpt-oss-120b -- are now available as NVIDIA NIM microservices, making it easy to deploy them on any GPU-accelerated infrastructure with flexibility, data privacy, and enterprise-grade security. Developers – you can also try out a preview of the gpt-oss models for free using our preview API and the web playground environment.

76