Chandrasekar Srinivasan

Today in #SiliconValley: NVIDIA spent more than $900 million to hire Enfabrica CEO (and an ex Broadcom employee) Rochan Sankar and other Enfabrica employees at the AI startup and to license the company’s technology, CNBC reported: https://lnkd.in/gCKQJGYp David Vellante John Furrier Robert Hof Crawford Del Prete Stacy Rasgon

378

🖥️ 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗠𝗖𝗣? 𝗧𝗵𝗲 𝗨𝗦𝗕-𝗖 𝗼𝗳 𝗔𝗜 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀 MCP (Model Context Protocol) is an open standard that helps AI applications easily connect with different data sources and tools—just like a USB-C port connects devices to accessories. 🔍 𝗪𝗵𝘆 𝗠𝗖𝗣? LLMs (Large Language Models) need to interact with different data and tools to work effectively. MCP makes this easy by providing: 🔹 📦 Pre-built integrations – Plug your LLM into a growing list of tools. 🔹 🔄 Flexibility – Switch between LLM providers without major changes. 🔹 🔒 Security – Keep your data safe while using AI in your own infrastructure. ⚙️ 𝗠𝗖𝗣 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 MCP follows a client-server model where applications (clients) talk to data sources (servers) securely. 🔗 Key Components: 1️⃣ MCP Hosts – AI tools like Claude Desktop, IDEs, and other programs that request data. 2️⃣ MCP Clients – Connect hosts to MCP servers. 3️⃣ MCP Servers – Lightweight programs that provide specific data or tools to the AI. 4️⃣ 📁 Local Data Sources – Your computer’s files, databases, and internal services. 5️⃣ 🌐 Remote Services – External APIs and cloud-based tools. MCP enables smooth communication between these components, ensuring AI models have the right context to work efficiently. 🎯 𝗧𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗠𝗖𝗣 MCP makes AI applications more powerful, flexible, and secure. As the ecosystem grows, more integrations will be available, making it easier for AI to understand and interact with the world around it. 💡 𝗧𝗵𝗶𝗻𝗸 𝗼𝗳 𝗠𝗖𝗣 𝗮𝘀 𝘁𝗵𝗲 𝘂𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹 𝗰𝗼𝗻𝗻𝗲𝗰𝘁𝗼𝗿 𝗳𝗼𝗿 𝗔𝗜 𝗮𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀—𝗷𝘂𝘀𝘁 𝗹𝗶𝗸𝗲 𝗨𝗦𝗕-𝗖 𝗳𝗼𝗿 𝗱𝗲𝘃𝗶𝗰𝗲𝘀! More - https://lnkd.in/gAEq_5fD #MCP #AI #LLM

110

36 Comments

Could state space models (SSMs) replace transformers as the dominant architecture for AI? Karan Goel , co-founder and CEO of Cartesia thinks so.  After completing his PhD at Stanford, Karan and his co-founder Albert Gu transformed their academic research on SSMs into Cartesia - starting with voice applications but with a much broader vision. Karan explains how SSMs process information sequentially rather than in batches, creating more memory-efficient systems that can adapt to new information in real-time - much closer to how humans process information. We also discuss: The differences between SSMs and transformers Why architectural innovation remains crucial alongside scaling The unique challenges and opportunities in voice AI His journey from academic researcher to startup founder The future of on-device AI powered by more efficient models Our full conversation here: https://lnkd.in/giidN5au

49

1 Comment

The deepened, strategic partnership between Arm and Meta is redefining #AI efficiency across every layer of compute. My team, along with others in our Central Engineering Software and Machine Learning teams, have been helping to enable Meta’s AI ranking and recommendation systems to run on Arm #Neoverse, delivering higher performance and lower power consumption than x86 systems. The collaboration also extends to Meta's foundational AI software technologies, like #PyTorch and #ExecuTorch, accelerating the ease of model deployment and increasing AI application #performance from #cloud to #edge. This marks the start of an even deeper collaboration between two technology leaders, paving the way for a more intelligent, efficient, and accessible AI future. (cc Milos Puzovic Chris Goodyer Allaukik Abhishek James Greenhalgh)

67

1 Comment

I have some personal news I'd like to share. I started an AI company with my dear friend and college roommate, Wayne Pan. It took over two decades, but everything has fallen into place, personally and professionally. At Cisco I led edge compute in industrial IoT.  At NEOM I helped shape nation-building technology strategies.  At SambaNova I scaled AI infrastructure globally and built deep partnerships across the Americas and the Middle East. That journey taught me something simple: AI is no longer just an important piece of technology. It is national infrastructure. And that realization is what led me here, to building haimaker. And before I give you the pitch, I wanted to say a few things. → Get your children involved in AI. Teach them how to use the tools. Let them use it to express their creative side. Or maybe show off their tech-y side. And maybe teaching is the wrong word, because in many respects, we’re all still learning. AI is a superhuman ability. Let’s get our kids involved in using these tools. → We are living through one of the greatest technological innovations of our lifetime. It’s a blessing to be able to witness the impact that AI has had on my life and career. I’ve never felt the vibe like this before. It’s something bigger than all of us. AI brings out the little kid in me a lot of times. It’s fun to tinker and create new things with the endless tools that are available. → We are so early. So, so, so early. When I think of AI. I think of what’s possible. I find myself discovering new things every day that I can’t believe are possible because of AI. I don’t think we are anywhere near our limit of discoveries. Don’t ever feel that you’re too late to get involved with AI as a career path. → AI isn’t a feature. It’s critical national infrastructure. Just like airports, hospitals, and telco networks, AI is essential to the growth and security of every nation. And it should be treated as such. Back to haimaker. haimaker is a Sovereign AI Compute Orchestration Platform. haimaker enables nations and telcos to launch sovereign AI Compute as a Service, while giving enterprises and developers the ability to run workloads in compliant regions. This is just the beginning. We have a lot of great news coming very soon and we can’t wait to share what we’ve been working on.

466

116 Comments

NVIDIA announces support for DeepSeek-R1. NVIDIA is actually a beneficiary of this wave of disruption, so I remain optimistic about NV. The real impact is felt by companies like OpenAI or Anthropic that rely on single closed-source models. The key point is that DeepSeek is open source and can be deployed at low cost - everyone can take it, modify it, and use it, including American companies, which have indeed quickly followed suit.

112

7 Comments

Ditto in Action: A Success Story of Real-World Adoption of NVIDIA Torch-TensorRT The latest NVIDIA blog highlights how Torch-TensorRT is enabling fast and efficient LLM inference in production, through the open-source framework Ditto developed by SqueezeBits. By effortlessly converting Hugging Face models into optimized TensorRT engines, Ditto demonstrates the practical value of Torch-TensorRT for accelerating deployment without compromising flexibility or usability. 🔗 GitHub: https://lnkd.in/ge8CQDGp 🔗 Blog: https://lnkd.in/gSUwn-N2

38

1 Comment

I went from Senior PM to VP in less than 3 years. If I could do it all over again, here’s what I would change. I was a Sr. Director, and the Global Head of DEI for one of the world’s leading EDA companies, a top innovator in chip design. But eventually, I started to feel like I was hired to be the face of the function, and not the actual leader of the function. Reflecting on that experience as that company's first-ever DEI leader, there were natural growing pains I probably should have stuck around for. At the same time, another company came calling with a VP title, the highest base salary I’d ever seen, a 40% bonus, and a very nice stock package. It felt like a no-brainer. But here’s what happened: I left because I felt handcuffed in my Sr. Director role, and I let the money and title at the new company make the decision for me. What I didn’t do? The simplest thing. I didn’t check the company’s financial health. If I had just looked at their balance sheet or quarterly reports, I would have seen the red flags. Instead, I got caught up in the base pay and made a bonehead decision. The result? The 40% bonus came in the form of 10% and as bad as we were doing financially, I was thankful for that. The stock??? It dwindled down to nothing, eventually forcing the company to go private. And after I was there for a year and a half, the company laid off two-thirds of its workforce, and I was part of the fallout. The hardest part? If I had stayed at my previous company, I’d be a millionaire today just from the stock alone. Here’s what I learned: - Don’t just chase the salary, study the stability. - Culture and values outlast compensation packages. - Protect your long-term wealth, not just your short-term income. If you’re considering a new opportunity, don’t make the mistake I did. Look past the base pay and into the financial facts and future of the company! They say hindsight is 20/20, so I would love to hear from you. What’s one career decision you wish you could do over again and why?

54

Yesterday, I shared the big step forward with Cerebras Systems integration into Meta’s flagship new Llama API. What’s in it for developers? 1/ The fastest Llama 4 inference in the world. —18x faster than GPUs, 4x faster than the next best 2/ Full accuracy — native 8/16-bit, no compromises 3/Easy migration — 1-click API and 3 code changes to switch from OpenAI 4/ No lock-in — prompts and outputs remain yours, with full provider portability 5/ Production-ready — consistent, predictable speeds at scale Can’t wait to see the innovations that the AI community will unlock 🔓 ⚙️ 👊

277

5 Comments

Golden Handcuffs, AI Hardware Edition A friend of mine, a senior engineer at one of the big AI chipmakers, recently told me he feels “stuck.” Not because he dislikes his work, but because, as he put it, “I’m sitting on a small fortune that keeps vesting every quarter. I’d be crazy to leave.” That’s the modern version of golden handcuffs. And this time, they’re not coming from Big Tech. They’re coming from AI chipmakers. In the past, this phenomenon was mostly associated with FAANG engineers. Explosive stock growth meant their equity packages became so valuable that no other company could realistically compete. If your Meta or Google stock had 4x’d, even a great new offer elsewhere looked small by comparison. But now, the tables have turned. The hottest “golden handcuffs” in tech aren’t being forged in Silicon Valley’s software giants. They’re coming from high growth private companies and the AI hardware supply chain. Companies like AMD, Broadcom, and NVIDIA have quietly minted some of the most competitively-paid engineers in the industry, not through mega signing bonuses, but through runaway stock growth. We pulled a few recent offer data points from Levels.fyi. Real compensation packages engineers received at these companies in late 2023, and compared what those same grants are worth today. Here’s what we found: AMD Senior SWE: ~$230K → ~$278K (↑21%) Broadcom ICB 4: ~$343K → ~$576K (↑68%) Nvidia IC 4: ~$360K → ~$670K (↑86%) (Assumes even vesting across four years for each offer.) These aren’t new offers, they’re the same packages, just repriced using current stock values. The result? Even modest equity grants have turned into small fortunes, and for many engineers, leaving now means walking away from hundreds of thousands (or millions) still left to vest. It’s a subtle but powerful form of retention. In the past, we’ve seen AI labs and Big Tech giants deploy explicit retention tactics, multi-year “stay bonuses,” or seven-figure RSU refreshes to keep their top talent from being poached. But for AI chipmakers, the retention power is already baked into the stock. Their share prices have become their most effective golden handcuffs. And as one hardware engineer put it: “My stock doubled in a year. I’m not leaving until this finishes vesting. Even if I wanted to.” The question now is what happens next. As these four-year grants start to fully vest, will we see a new wave of engineers finally cash out and move on? Or will the next run-up in AI chip stocks lock in a new generation of talent just as tightly? Either way, the golden handcuffs have changed hands. And this time, they’re made of silicon. Links to offers used to model this chart: AMD: https://lnkd.in/gYFQ4pR9 Broadcom: https://lnkd.in/gBX4CqKX Nvidia: https://lnkd.in/gDKxtm7X

134

8 Comments

Following up on the AWS launch blog — here’s our internal view on why this work matters. NVIDIA Dynamo is now optimized for Amazon EKS, EC2, and SageMaker, enabling cost-efficient, high-throughput inference for large models using TRT-LLM, vLLM, and more. For our product teams, AWS is a critical platform to deliver scalable, real-world impact. This blog outlines how we’re designing Dynamo to meet the demands of production GenAI deployments running on AWS infrastructure. Proud to contribute to this momentum and support the engineering teams bringing it to life. 🔗 https://lnkd.in/gNYc_cAj Amr Elmeleegy Harry Kim

38

When it comes to autonomous driving, Tesla and Waymo couldn’t be more different in how they approach the problem, and what they believe it will take to solve it. Waymo is all-in on the “hardware-heavy” school of thought: LiDAR, radar, HD maps, redundant compute, and highly curated geo-fenced environments. It’s a masterpiece of engineering, no doubt. And it works, but only in specific cities, under tightly controlled conditions, and with a very high operational cost. Tesla, on the other hand, is betting on vision and scale. No LiDAR. No HD maps. Just neural networks trained on billions of real-world miles, constantly updated from a global fleet of cars. Less hardware. Lower cost. More scale. And most importantly, a direct path to consumer vehicles, not just robotaxi pilots. From a tech perspective, the contrast is clear: • Waymo builds autonomy like it’s building a #NASA mission. • Tesla builds it like it’s scaling a #smartphone feature. I’m leaning toward Tesla’s approach, not because it’s perfect, but because it’s data-rich, less dependent on expensive hardware, and more #economically viable in the long run. Of course, the race isn’t over. #Regulations, edge cases, and safety validation will shape the outcome. But if autonomy is ultimately a software problem at scale, I think Tesla may be closer than we think. HITEC Angeles Investors The Tech Series Porsche AG Pirelli NASA - National Aeronautics and Space Administration General Motors Rivian BYD U.S. Department of Transportation OpenAI

349

326 Comments

https://lnkd.in/gfAvsvAn LLMs are not just good at text-to-text regression. In this blog post, Yash Akhauri and Xingyou Song talk about how regression language models can also be used to *simulate large warehouse-scale systems* like the Google fleet, predicting performance of complex systems *directly from raw text data*.

146

1 Comment

There’s something almost poetic about an #analogchip startup called Blumind going full throttle in the world of low-power silicon. Because while the tech world’s been overdosing on #AIhype and #GPUhoarding, Blumind built something that doesn’t just nod at the future, it stares it down and whispers watch this at under a #milliwatt. Founded in 2020 (not 2023, let’s get the receipts right), Blumind is based in Ottawa and led by Niraj Mathur, a 25-year #semiconductor vet who’s stitched his name into the boards at Nortel, Rambus, and Semtech. Alongside John Gosson, a co-founder with #neuralnetwork street cred that predates your first TikTok algorithm, they’ve been engineering a brainchild that doesn’t just scale AI, it strips it back to its most elegant, efficient form. Their AMPL platform isn’t a chip trying to squeeze into the AI conversation. It’s an analog assassin built for event-driven #neuromorphiccompute, no ADCs, no DACs, no excuses. Think #biology channeled through #silicon swagger. We’re talking #submillisecond #latency and power profiles that make today’s #ASIC dinosaurs look like coal furnaces in a lithium world. They’re not stuck in R&D limbo either. This thing tapes out on standard 28nm–65nm #CMOScompatibility. Translation: it’s real, it’s compatible, and it’s on its way. And now? $20M CAD in fresh Series A firepower, co-led by Cycle Capital and BDC Capital, with Fusion Fund, Two Small Fish Ventures, and Real Ventures in the mix. Add in earlier rounds through Creative Destruction Lab and Venturelab, and we’re at $23M CAD raised. But this isn’t just capital, it’s conviction. A bet on a team rewriting the way chips sip power instead of chugging it. Blumind isn’t chasing benchmarks. First-gen #sensorprocessors. Then chips for #vision and #largelanguagemodels. All while expanding their engineering muscle across Ottawa and Toronto and locking in tier-1 traction across #industrial, #mobility, and #medtech. And this isn’t some hypothetical science fair. Blumind’s chips don’t sip power, they breathe it. We’re talking multi-year #batterylife on #edgedevices that used to croak after a weekend. That’s a game-changer for #AIoT, #mobilitysafetysystems, and #medicalsensors where latency isn’t just lag, it’s life or death. While everyone else scales #datacenters like suburbs, Blumind’s making #edgeAI actually sustainable. And no, they didn’t pull up at #CES2025 for lanyards and lunch. They came with eval kits and walked out on a “Top 10 AI Hardware Startups” list. That’s not a booth visit. That’s a statement. Props to Niraj Mathur and John Gosson for keeping it heads down while building something this loud. Blumind isn’t trying to out-GPU the GPU crowd. They’re redefining what power means, with analog elegance and disruptive clarity. #Startups #StartupFunding #VentureCapital #CanadaTech #Semiconductors #GridTech #DeepTech #EdgeComputing #AIHardware #TechEcosystem #StartupEcosystem

28

2 Comments

Efficient LLM Inference: Bandwidth, Compute, Synchronization, and Capacity are all you need In the world of large language models (LLMs), memory bandwidth and capacity are pivotal for efficient inference workloads. Discover how optimizing these elements can significantly boost per-user throughput and drive system-level efficiency. Key Takeaways: - Serving LLMs demands 100s of GB per server for model instances, highlighting the importance of memory capacity. - High memory bandwidth is crucial for maximizing throughput. - DRAM-based designs offer superior efficiency in terms of cost and power usage. These insights, drawn from a comprehensive study by NVIDIA Research, highlight the core performance bottlenecks and potential hardware advancements that can redefine LLM deployment strategies. Read the full article here: https://lnkd.in/gNhFm8Ki #AI #LLM #MemoryBandwidth #InferenceWorkloads

18

1 Comment