Bhavana Chandrashekhar

Bezos's Prometheus Raises $12B for Physical AI A $12B round at a ~$41B valuation takes Project Prometheus past $18B total. What the physical-AI capital wave means for India and UK robotics, simulation and hardware builders. Read → https://lnkd.in/gRwbGd-n #AINews #ArtificialIntelligence #AIBuilders #VerifiedBuilders

2

We got 5–10× more throughput from an 8GB GPU. No new model. No new hardware. Achuth Reddy Bangaru and I spent the past few weeks treating LLM inference as a systems engineering problem, not a modeling one. The question: how much performance is left on the table when you ignore serving strategy? A lot, it turns out. The Setup Single NVIDIA RTX 4060 (8GB). TinyLlama-1.1B. Three serving strategies built from scratch using FastAPI + asyncio: Baseline (single-request), Static batching (fixed batch of 4), Dynamic batching (async queue, 20ms window, adaptive 1–4 batch) Custom load generator (httpx + asyncio) simulated light, medium, and heavy traffic. Every run measured req/s, p50/p95 latency, tokens/sec, and GPU utilization. 📊 Results at medium load (30 req, concurrency 5, 64 tokens): Baseline — 0.57 req/s | 8.3s p50 | ~25% GPU Static Batching — 0.71 req/s | 6.2s p50 | ~60% GPU Dynamic Batching — 1.73 req/s | 2.2s p50 | ~75% GPU 💰 Cost per 1,000 requests on a $1/hr GPU: Baseline ~$0.55 → Dynamic ~$0.17 Same hardware. 3× the throughput. 68% cost reduction. 💡 The Real Insight The baseline GPU was sitting at ~25% utilization. Not because the hardware was weak, because the serving strategy was naive. One request at a time means the GPU idles between calls, KV cache goes unused, and you're paying for compute you're not using. Dynamic batching fixes this by queuing requests in a short time window (20ms), adaptively grouping 1–4 requests per forward pass, and keeping the GPU fed continuously. The result isn't just better benchmarks, it's directly lower cost per user in production. What's in the Repo Baseline, static, and dynamic servers (fully reproducible), custom async load generator with p50/p95 metrics, KV cache experiment, interactive 2D visualization showing execution flow across all three strategies, and a full benchmark report with charts. If you're thinking about LLM deployment cost, async scheduling, or just want a hands-on systems lab, worth a look. 🔗 https://lnkd.in/e7SxnGqV Happy to discuss what we'd tackle next, continuous batching, PagedAttention, speculative decoding. #LLMInference #MLOps #AIInfrastructure #SystemsEngineering #GPUOptimization #MLEngineer #AIEngineer

28

3 Comments

Last few days have been running ForgeCAD CAD-as-code system as a benchmark on most major AI models with same prompt to build an AC system. The results were somewhat surprising. While many new models compete on existing benchmarks and show results matching or beating the strongest models, here the results were less saturated. Only a few models continuously performed good. Opus 4.5, Opus 4.6, GPT-5.2-Codex, GPT-5.3-Codex, and surprisingly Gemini-3-Flash (for not hard models or low complexity changes). Most other models I tried had difficulties with spatial reasoning. With GPT-5.3-Codex I took it further, asking for harder and harder models, so far the capabilities are very strong. I shared the results of all tested models in the GitHub repo: https://lnkd.in/dm5SmCYT

17

Thank you to Vimal Patel and iMerit Technology for the conversation. If you are attending ECCV 2026, join us for the 10th AI City Challenge Workshop on September 8, from 8:30 AM to 12:30 PM at Malmö Arena Terassen. This anniversary edition brought together 325 registered teams from 26 countries and regions, representing nearly 33% growth over last year. The workshop is organized through a global collaboration involving NVIDIA, Woven by Toyota, Milestone Systems, Santa Clara University, University at Albany, Iowa State University, and other academic and industry partners. Organizers and track leaders include Rama Chellappa, Zhedong Zheng, Renran Tian, Judah Goldfeder, and the broader AI City Challenge team. The program will showcase leading work in multi-camera 3D perception, transportation safety, traffic anomaly reasoning, text-based person re-identification, generative traffic video forecasting, and cross-city object detection, along with two out-of-domain evaluations. Come hear directly from top-performing teams, explore open-source methods, meet researchers at the poster session, and see the challenge winners announced. Workshop program (including Zoom info): https://lnkd.in/g4WtfgYh #ECCV2026 #AICityChallenge #ComputerVision #PhysicalAI #Sim2Real

31

2 Comments

If you can reliably detect a pattern in messy data, you can build half of modern tech. 🛜 One of the core ideas that makes this possible: Matched Filtering. Most sensing and receiver systems boil down to one question: “Is the pattern I care about hiding in this messy data?” So what is a matched filter? You take the waveform you expect (your “template”) and you slide it across your measured data, scoring how well it matches at each time offset. ⌛ That score is a correlation. When the template lines up with the real signal, you get a big peak. Peak = “it’s here.” Peak location = “this is when it arrived.” Why does this work? 🤔 : Noise is unstructured. It doesn’t line up consistently. Matched filtering performs coherent accumulation: it aligns the received data with the expected waveform and sums the aligned contributions. The signal adds constructively and uncorrelated noise adds inconsistently, so its net contribution grows much more slowly. Where you’ve seen it (even if you didn’t call it that) 👉 GPS/GNSS: the receiver acquires and tracks satellites by correlating the incoming signal with locally generated PRN codes (and searching over code phase and Doppler). 👉 Radar/Sonar: echoes are detected and range-resolved by correlating the received return with a reference of the transmitted waveform (pulse compression / matched filtering), often followed by thresholding (ex: CFAR). 👉 Wireless: receivers use matched filters/correlators for optimal detection of known pulse shapes and training/preamble sequences. In spread-spectrum, despreading is correlation with the spreading code. Synchronization often uses correlation with known sequences. 👉 Time-delay estimation (audio + sensors): time-of-arrival and echo delay are estimated via cross-correlation / generalized cross-correlation between signals or between a signal and a reference template (used in alignment, localization, and echo finding). Where have you used correlation / matched filtering in real life? GPS, radar, comms, audio… or something unexpected?

29

Powerful takeaways from today’s “Building AI Agents for Enterprise” session with Ajay Shenoy and Abhijith Neerkaje The discussion went well beyond surface-level RAG patterns. We unpacked Retrieval-Augmented Generation from first principles through production realities—why chunking is not a preprocessing detail but a design decision, how embedding spaces influence recall vs. precision, the role of retrieval strategies under latency constraints, and why evaluation frameworks are the hardest (and most critical) part of making RAG reliable in production. A key reminder: fundamentals scale. These concepts aren’t academic—they directly determine whether AI agents can reason autonomously, retrieve the right context under ambiguity, and make defensible decisions in enterprise systems where correctness, cost, and observability matter. The Q&A stood out. Seeing seasoned practitioners deconstruct real-world failure modes—hallucinations, poor grounding, retrieval drift, and evaluation blind spots—and reason through solutions live was invaluable. This is what effective technical learning looks like: clarity rooted in experience, not abstractions. Agentic AI is no longer a future concept. As systems evolve from prompt-response to planning, tool-use, memory, and self-correction, a solid grasp of RAG fundamentals becomes non-negotiable for anyone building production-grade AI. Grateful for sessions like this—where theory meets operational reality, and learning is anchored in what actually works at scale. #AgenticAI, #LLM,#RAG, #AgenticRAG

25

2 Comments

For a long time, this was the assumption You need 24GB+ VRAM to even experiment with large LLMs. That assumption is changing. With memory-optimized inference, 70B parameter models can now run inference on 4GB GPUs by loading and executing one layer at a time instead of keeping the full model in memory. What’s happening under the hood Load a layer. Run inference. Release memory. Repeat. There are trade-offs. Higher latency, heavy CPU and RAM usage, and this is inference only. But it dramatically lowers the barrier for local testing and experimentation. Why this matters • Prototype large models without cloud GPUs • Test agent workflows locally • Explore Llama, Qwen, Mistral AI, Mixtral-class models on modest hardware • More builders get hands-on access to large models Projects like AirLLM show how much headroom still exists in inference optimization. https://lnkd.in/g-DHRkk9 #AI #LLMs #OpenSource #MachineLearning #GenAI #Developers #Builders

25

1 Comment

Workshop: Amazon AI Engineering Blueprint 🚀 To truly innovate in the AI space, you have to understand the exact architecture, high-level system designs, and production standards used by industry leaders. This deep dive laid out the exact parameters required to engineer and scale AI systems for global infrastructure. Huge thanks to Ankush Singla and Coding Ninjas for sharing these advanced insights and strategic frameworks. Time to put these models into production! 🔥 #AI #MachineLearning #CodingNinjas #Amazon #DeepLearning #SystemDesign #SoftwareEngineering #DataScience #CloudComputing #TechInnovation

2