Andy Xin

AI’s best-kept secret: the smartest models are hopelessly hardware-obsessed. 🔹 Speed → attention kernels hand-tuned for specific silicon 🔹 Moat → vertical stacks (TPU, Dojo, H100) that outsiders can’t replicate 🔹 Risk → betting on generic GPUs while rivals optimize end-to-end I break down why “tight coupling” is reshuffling the AI power map—and how open-source can close the gap. Read the full analysis ⤵️⤵️ https://lnkd.in/dTbrvh68

1

semianalysis benchmarked ~1000 NVIDIA and AMD GPUs. GB300 NVL72 gives 100x better inference perf than H100. here's the main takeaways from the newly released InferenceX: all inference economics reduce to one curve: throughput vs interactivity (tokens/sec/user). high batch = high total throughput, low cost per token, but each user gets tokens slower. low batch = fast per-user responses, but fewer total tokens and higher cost per token. every result in this benchmark is a point on this curve. a useful analogy here is that of a bus v.s. an f1 car. the bus can transport more people but obviously goes much slower. while the f1 car only fits 1 person but can go much faster than the bus. every frontier lab (openai, anthropic, xai, deepseek) runs disaggregated prefill + wide expert parallelism + FP4. disagg separates prefill (compute-heavy, processes all input tokens at once) from decode (memory-heavy, generates one token at a time) onto separate GPU pools. this eliminates interference between the two phases. wide EP distributes experts across many GPUs instead of replicating the whole model. deepseek R1 has 256 experts but only 8 activate per token. with EP64 across 64 GPUs, each GPU holds just 4 experts instead of 32, freeing HBM for KV cache and increasing arithmetic intensity because 8x more tokens flow through each expert. jensen underpromised on blackwell. at GTC 2024 he claimed 30x inference gains from H100 to GB200 NVL72. everyone called it "jensen math". InferenceX actually measured up to 100x on GB300 NVL72 FP4 vs H100 disagg+wideEP FP8. even cost-adjusted: up to 65x better tokens per dollar. they had to add a log scale to the dashboard to visualize it. NVL72 is a large unlock. 72 GPUs connected at 900 GB/s via NVLink vs 8 GPUs per B200 node over InfiniBand at ~100 GB/s. wide EP across 72 GPUs without ever leaving the high-bandwidth domain. the AMD most recent story is a composability problem. MI355X single node FP8 SGLang actually beats B200 SGLang at certain interactivity levels. the hardware is competitive. but compose FP4 + disagg + wideEP together (what every lab actually deploys) and MI355X gets mogged. individual optimizations work in isolation but fall apart when stacked. AMD's ATOM inference engine has zero production customers. MI355X has zero tests on vLLM CI. MTP is probably the largest inference optimization right now. multi-token prediction uses built-in auxiliary heads (no separate draft model) to propose multiple tokens per step. on GB300 FP4, MTP drops cost from $2.35 to $0.11 per million tokens at 150 tok/s/user. 21x cheaper from one flag. works across all SKUs with no meaningful accuracy loss. what surprised you most from InferenceX v2? read the full article: https://lnkd.in/gGwpS4ur

138

2 Comments

The industry is trying to solve the AI scaling problem with more FLOPs and more HBM. But the real bottleneck isn’t compute. It’s memory orchestration. LLMs spend most of their time waiting for data movement rather than doing useful work. This is why architectures that tightly couple memory and compute rather than separating them are becoming increasingly important. At JSA LAABS Private Limited , we have been exploring this problem through a different lens: cognitive compute architectures where sparsity, temporal dynamics, and memory locality are native to the hardware. The next wave of AI infrastructure may not be about larger GPUs. It may be about smarter silicon. #AIInfrastructure #Semiconductors #AIHardware #AcceleratedComputing #EdgeAI #ArtificialIntelligence #ChipDesign

11

1 Comment

I was asked recently: what is one thing that has surprised me about leadership? Now, keep in mind I went to Stanford to be a great engineer, and I started at Intel in the engineering department. Engineering was where I lived. As I moved up within the organization, I had to start spending time with other areas of the business. Departments like marketing, business, legal, communications, etc. Guess which one I spent the most time in? Trick question -- it was the psychology department! A huge part of a leader’s job is seeking to understand and influence people. It's one of the most important skills in this role. I would never have believed this as an 18-year-old engineer! Some of the writings and efforts of Pat Lencioni, the The Table Group and the Five Dysfunctions of a team have been really helpful to me over the years to skill up in this area. 

887

43 Comments

🚨JUST IN: OpenAI's $10B Cerebras deal signals AI hardware shakeup: Nvidia's GPU reign, now challenged by wafer-scale disruptors OpenAI just inked a $10 BILLION deal with Cerebras—disrupting the GPU status quo. The AI arms race just found a new supplier. Cerebras' wafer-scale chips will power OpenAI models for faster, more complex workloads. This is what you need to know: •⁠ ⁠$10B deal dwarfs prior hardware contracts •⁠ ⁠Nvidia's dominance faces its first real threat •⁠ ⁠AI infrastructure wars just escalated This means next-gen AI may be built on non-Nvidia silicon. Will startups and enterprises rush to dump GPUs for new AI chips?

20

1 Comment

New on Vidhai — Nvidia Moat The Moat Behind the Magic: Nvidia's edge is not just silicon. It is a layered moat built from CUDA, supply chain control, cloud financing, and geopolitics — all compounding at once.Nvidia's job is simpl... Read the full article → https://lnkd.in/gJtvMBCj #AI #Vidhai #TechInsights #Nvidia

3

Jensen Huang's keynote talk in the 1st GTC conference in 2009 The first GPU Technology Conference (GTC), held in September 2009 at the Fairmont San Jose, was a pivotal moment for NVIDIA. It marked the company’s official transition from being a "graphics company" for gamers to a "parallel computing company" for scientists and researchers. At the time, Jensen Huang delivered a keynote that was as much about a new philosophy of computing as it was about new hardware. Here is a summary of the core themes and announcements: 1. The Debut of the "Fermi" Architecture The biggest technical reveal was the Fermi microarchitecture. This was NVIDIA’s first GPU designed from the ground up for general-purpose computing (GPGPU) rather than just rendering pixels. Key Features: It introduced crucial features for scientific computing, such as L1/L2 caches, ECC memory error protection (vital for long-running simulations), and significantly improved double-precision floating-point performance. The Vision: Jensen framed Fermi as the world’s first "computational GPU," capable of handling tasks that were previously the exclusive domain of expensive CPUs and supercomputers. 2. "Heterogeneous Computing" Jensen spent a significant portion of his talk explaining the concept of Heterogeneous Computing—the idea that the CPU and GPU should work together. The Analogy: He argued that CPUs are great for "sequential" tasks (logic, OS management), while GPUs are built for "parallel" tasks (mathematical simulations, data processing). Efficiency: He demonstrated how a parallel application could be sped up by 100x to 200x using a GPU, while the system still relied on the CPU for its traditional strengths. 3. Real-World Applications (Scientific Computing) To prove that GPUs weren't just for Crysis, Jensen showcased demos that had nothing to do with gaming: N-Body Simulations: A demo showing thousands of interacting particles to simulate galaxy formations or molecular dynamics. Medical Imaging: Showcasing how GPUs could reconstruct 3D images from CT scans in real-time. Ray Tracing: Early demonstrations of interactive ray tracing, hinting at the future of professional visualization. 4. Investing in the Midst of a Crisis The 2009 GTC took place just as the world was recovering from the 2008 global financial crisis. Jensen famously used the talk to emphasize NVIDIA's resilience. He noted that while other companies were cutting back, NVIDIA was doubling down on R&D. "I'm going to invest more in R&D this year than last year because I believe in the future of the GPU." 5. Why it Matters Today Looking back, the 2009 keynote is considered the "seed" of the modern AI revolution. By convincing the world that GPUs could be used for general math, NVIDIA paved the way for researchers to eventually run deep learning models on CUDA-enabled hardware—the very same technology that powers ChatGPT and modern AI today.

221

40 Comments

Why NVIDIA DeepStream SDK is Essential for Real-Time Computer Vision Struggling to scale your computer vision application across multiple video streams while maintaining low latency? NVIDIA DeepStream SDK is built exactly for this challenge. DeepStream is a comprehensive real-time streaming analytics toolkit built on GStreamer for AI-based multi-sensor processing of video, audio, and image data. It includes over 40 hardware-accelerated plugins and 30 sample applications, providing ready-made building blocks instead of building from scratch. It delivers complete end-to-end GPU acceleration for the entire video processing pipeline. Example: RT-DETR with C-RADIO-Base on an L40S GPU processes 609 streams of 1080p/30fps video with full object detection and tracking. Key Advantages: ✅ Deploy Anywhere: Build once and deploy on cloud, workstations, or Jetson edge devices with off-the-shelf containers and Kubernetes support. ✅ Rapid Development: DeepStream Inference Builder enables deployment in minutes from YAML configuration, automating data flow, preprocessing, and model execution. ✅ Advanced Features: Multiview 3D tracking across camera networks with automatic ID assignment and identity preservation through occlusions. ✅ Flexible Integration: Native support for Triton Inference Server allows deployment in PyTorch/TensorFlow for rapid iteration or TensorRT for maximum production performance. It removes the complexity of video pipeline optimization, letting you focus on building unique value. Whether you're working on smart cities, retail analytics, industrial inspection, or healthcare monitoring, It accelerates your path from prototype to production. ♻️ Share this to help others learn computer vision, deep learning, and vision language models. 👉 Check the comments for resources that will help you start learning DeepStream. #computervision #deeplearning #cuda #gpu

37

5 Comments

 🚀 C++ Face Swap Swapping faces using C++ and standard logic. No heavy external AI libraries just pure pixel manipulation and math. ✔️ Calculate average RGB shifts to blend skin tones. ✔️ Use distance-based interpolation for "soft" edges. ✔️ Real-time adjustment using keyboard arrows for pixel-perfect alignment. ✔️ A lightweight, fast, and manual way to merge images with professional-looking transitions . 👀 code: https://lnkd.in/g-NXV-iX #cpp #programming #faceswap #sfml #gamedev #coding #algorithms #imageprocessing #softwareengineering #math #blending #sourcecode #cpp23 #sfml3 #graphics

48

Nvidia's roadmap is set. The AI chip market is shifting. The real story isn't just about the next chip—it's about the entire ecosystem holding it all together. At GTC 2026, Jensen Huang will unveil the plan to defend Nvidia's dominance. The focus is clear: inference workloads, agentic AI, and the data center. But the landscape has changed. While Nvidia still commands over 90% of the market, the pressure is building from all sides. 👉 Major customers like OpenAI and Meta are building their own chips. 👉 Competitors are closing the gap on performance and cost. 👉 The shift to specialized, application-specific chips is accelerating. Nvidia's response is a masterclass in strategic defense. They aren't just iterating on silicon. They're building an impenetrable fortress. 💡 The $17B acquisition of Groq wasn't just about buying a fast inference chip. It was about integrating that speed directly into the CUDA ecosystem. 💡 The $2B investment in laser-optics firms isn't a side project. It's the key to unlocking chip-to-chip communication at a massive scale. The goal? To make leaving their platform more painful than staying. This is the new battleground. It’s no longer just about who has the fastest chip. It’s about who controls the most efficient, interconnected, and indispensable system. For anyone in tech, the lesson is universal: Your moat isn't your product. It's the entire experience and infrastructure you wrap around it. What's the one ecosystem you rely on that would be hardest to leave? #NvidiaGTC #AIHardware #TechStrategy #BusinessOfAI 𝐒𝐨𝐮𝐫𝐜𝐞: https://lnkd.in/gyhR5_rb

3

1 Comment

With NVIDIA's next-generation Rubin platform entering mass production, industry sources indicate that the company is accelerating the development of its next-generation Rubin Ultra platform, with design finalization (tape-out) expected in the second quarter of this year. The platform will continue with the 3nm process, but will bring disruptive changes to back-end packaging and PCB (printed circuit board) technology, adopting the CoWoP architecture for the first time. NVIDIA is collaborating with Taiwanese supply chain companies such as Siliconware Precision Industries (SPIL) and Zhen Ding Technology to advance pilot production and sampling, breaking through existing technological bottlenecks to meet the high-performance demands of future AI inference. SPIL has become a core partner in the implementation of the CoWoP architecture. It is understood that CoWoP-related production equipment will be installed at SPIL's Taichung plant as early as the first quarter of this year, with SPIL exclusively responsible for the pilot production of the CoWoP architecture, expected to begin in the second quarter of this year. https://lnkd.in/gdmRfUFd

14

AI runs on GPUs. The physical world runs on decisions. At NVIDIA GTC this week. As AI moves into real-world systems, a lot of what determines outcomes comes down to hardware programs - and the decisions behind them: • component choices • alternates • lifecycle and supply signals • cost-performance tradeoffs Wizerr AI team is around this week connecting with engineers, builders, and hardware teams thinking about this layer. If you’re at GTC this week and thinking about how AI connects to real-world systems, would love to compare notes. (Something we’ve been working on - more on Thursday.) #GTC2026

49

Are mixture of expert (MOE) models the future for professionals using local LLMs? Buying enough VRAM for 30-100B MOEs on a Strix Halo mini PC is quite affordable, and the inference speed for the 30B MOE is just awesome. Filmed on Ryzen AI MAX 395+ with 128GB RAM, Vulkan backend, using Lemonade: https://lnkd.in/eDZW7ZZn Great work by Daniel Holanda Noronha sprucing up the interface! Ramakrishnan Sivakumar Victoria Godsoe Tomasz Iniewicz Kalin Ovtcharov

58

2 Comments

🤖 A Waymo rider is born! 👩🏽‍🍼a mother in labor hailed a Waymo to the hospital in City and County of San Francisco and ended up with an extra surprise: her baby delivered in the backseat! This incredible moment is a testament to the trust riders placed in the physical AI robotaxi service. 2025 has been the year Waymo shifted from validation of the tech to begin the j-curve of scale. Key highlights of the year and what’s next: • massive scale: served over 14 million trips in 2025 alone, more than tripling public rides from last year. • global expansion: laying the groundwork to launch in over 20 new cities in 2026, including the first international expansions into #tokyo and #london. • safety first: bold commitment to safety resulted in a 10-fold reduction in serious injury or worse crashes compared to human drivers. • sustainability impact: the all-electric fleet helped riders avoid over 18 million kgs of Co2 emissions. •Rides scaled: Will grow from 1 million rides a month to 1 million rides per week in 2026! Follow me, Timothy Papandreou for the latest in emerging technologies #autonomousvehicles #waymo #ai #selfdriving #emergingtech #innovation

70

7 Comments

We are building our very own Vision Semantic Router (VSR) which will enable us to make every physical space programmable — this is very fun! 🙌

13

🕷️ SPIDER-MAN IS REAL — AND HE CLEANS SKYSCRAPERS 🏙️ At NVIDIA GTC in San Jose, I had a conversation with Ido Genosar from Verobotics that stopped me in my tracks. Not because of a flashy demo — but because of a very simple, very real problem nobody talks about. We build skyscrapers faster than ever. But maintaining them? Facade inspection and cleaning remains one of the most dangerous jobs in construction — and one of the most neglected markets in real estate. Verobotics is changing that with an autonomous robot that moves across vertical glass surfaces with the precision of a rock climber. The engineering behind it is what caught my eye: 🔩 a vacuum adhesion system that holds reliably on smooth facades under real-world conditions, while simultaneously capturing high-resolution data to feed digital twins. Maintenance is no longer reactive — it becomes data-driven. The business model is equally smart: Robot-as-a-Service — no CAPEX for building operators, no risk for rope-access teams, full scalability from Tel Aviv to San Jose. This is not a prototype for a trade show. 🏗️ This is physical AI becoming urban infrastructure. The vertical dimension of our cities has been ignored for too long — and the robots are finally coming for it. Will vertical robotics become the deciding factor in keeping our growing metropolises maintainable? Drop your take in the comments. 👇 Best regards Ulrich - The German Engineer #VerticalRobotics #PhysicalAI #NVIDIAGTC More Facts. More Automation. More Robotics. Less Show. Credit to Verobotics - at NVIDIA GTC / NVIDIA Robotics / NVIDIA Omniverse / NVIDIA Take a look at my own event HUMANOID EVOLUTION SUMMIT

44

7 Comments