Noel Campbell

I just found the fastest serving engine for LLMs: (fully open-source & makes vLLM 7x faster) LMCache is an LLM serving engine designed to reduce time-to-first-token and increase throughput, especially under long-context scenarios. At a high level, LMCache is like a CDN of KV caches. It sits between the LLM serving engine, like vLLM, and the external data by storing the KV caches of reusable texts across various locations like GPU, CPU, and Local Disk. This way, it can boost vLLM with 7x faster access to 100x more KV caches. Find a link to the GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

1,240

37 Comments

In distributed systems, reliability isn’t just about retries and durability, it’s about knowing who owns recovery. My latest post, based on the Coordinated Progress model I posted previously, explores how reliable triggers create responsibility boundaries and how those boundaries shape resilience, observability, and complexity. https://lnkd.in/diCmywnS

117

8 Comments

I've found that most devs have rarely looked at GPU compute dies. There's a tremendous amount of innovation that happens at the silicon level that can help in understanding optimization paths when considering different platforms. This blog, authored by Kyle Aubrey and Nick Stam, takes a deep dive into the Blackwell Ultra GPU architecture including the high-speed interconnects, SMs, and other critical components responsible for driving critical workload performance at scale. Blog link: https://lnkd.in/eUXNFb_m #nvidia #gpus #blackwell #hardware #ai

263

7 Comments

Seeing quite a few discussions lately about Kafka/Iceberg integrations being "zero-copy" or not. I think this is largely missing the point. First, where I agree is that this integration should be "zero-effort" for users. Materializing a Kafka topic into an Iceberg table shouldn't require more than a click of a button. Queries should provide a uniform way for accessing the data in both a topic and the corresponding table. This is the stream/table duality, and it should Just Work™. Now, whether this requires to store the bytes of data once in a Kafka topic, and a second time elsewhere for table access, shouldn't really matter from a user perspective. I'd argue storing the data twice is actually a benefit, and in fact it's a pattern well established: it resembles the design of WAL and table files known from databases for decades. I don't think anyone ever complained about this structure in their RDBMS? Which makes sense, it's an implementation detail, opaque to users. But as it turns out, having log and table data separately is even more advantageous for the deconstructed database that is Kafka and Iceberg: you can have multiple readers of the same log (Kafka topic), materializing views in multiple destinations and systems optimized for specific use cases. Maybe multiple Iceberg tables with different projections (think PII), maybe an Iceberg table and a full-text index in Elasticsearch, maybe an... you catch my drift. Furthermore, the log is replayable, so you can recreate views if needed, or you can implement new use cases you didn't originally have in mind. All in all, I think "zero copy" is mostly a red herring. Sure, it can be an optimization for certain scenarios, but mostly it's a distraction from the immense value you get from combining Kafka and Iceberg when done the right (seamless) way.

127

26 Comments

“In a very binary sense, Metronome made self-service at Anyscale possible.” That’s Matt Weber, Software Engineer, reflecting on Anyscale's expansion into new revenue channels. As the team added PLG and cloud marketplaces (AWS, GCP, and Azure), their manual billing setup couldn’t support the volume or complexity. Reporting, reconciliation, and manual workflows were slowing them down. With Metronome as their billing foundation, Anyscale unified every revenue motion on one platform and accelerated growth across the business. They launched self-serve billing end-to-end, brought AWS, GCP, and Azure marketplaces live in under three weeks each, and freed up three engineers and a PM from dedicated billing work. 🎉🤝 Big congratulations to the Anyscale team on the expansion—this is what it looks like when innovative product and modern monetization infrastructure come together. Read the full story at the link in the comments.

37

1 Comment

Some thoughts on Princeton’s recent results on superconducting qubits: 1. Amazing work from the team - this is the best work I’ve seen in 2025 for superconducting qubits. 2. A lot of scientists thought that there was some “magic” with tantalum on sapphire, with sapphire being the magic ingredient. Turns out it is the interfaces that matter most. 3. Being able to use high-resistivity Sillicon is important because most modern semiconductor manufacturing tools use Sillicon. This means that we can bring to bear all the advance manufacturing tools for semiconductors to build better qubits. 4. It isn’t clear that the magic ingredient is tantalum either. Aluminum is the bread and butter of most qubits. In fact- in this work, Al / AlOx junction was still used. However, to achieve the same results with Aluminum, careful processing is needed to reduce defects on the surface. 5. This work re-enforces Qolab’s strategy of using advance semiconductor manufacturing tools and interface engineering to make useful quantum computers. I want to thank Andrew Houck et al. for their amazing work. https://lnkd.in/e5g7ttFg

135

2 Comments

Great to see how a computational physics research lab is using Pulse's API to extract LaTeX equations and mathematical notation from their 40+ year archive of papers. Their team built a specialized pipeline that preserves the semantic structure between equations, preserving relationships between variables, theorems, and proofs. What's remarkable is how Pulse correctly identifies and extracts nested mathematical structures (matrices within summations within integrals) while maintaining proper symbol relationships. Our mission is to turn previously inaccessible computational knowledge into structured, searchable data that accelerates novel research. If you're a research team interested in trialing out Pulse, send me a DM for some trial credits! Sample LaTeX output attached below.

64

1 Comment

Karpathy redefines computing with LLMs as the new paradigm. CPU becomes LLM, bytes become tokens, RAM becomes context window. This conceptual shift positions large language models as an operating system (LMOS) rather than just applications running on traditional computing infrastructure.

74

6 Comments

Devin’s work at Bilt is equivalent to the work of ten engineers weekly, according to their own productivity metrics. With over 100 PRs merged weekly, Devin automates tedious Java version upgrades, investigates and fixes GCP issues, and accelerates software development across the board. Bilt CTO Kosta Krauth and the team’s formula for supercharging their engineering productivity with Devin: https://lnkd.in/gAKvSVRj

53

Thanks to the resurgence of RL, LLMs are finally able to reliably coordinate tools and reasoning to do high-precision retrieval. Companies like Happenstance, Clado, and Mintlify have already shifted to agentic search, and it's only a matter of time until anything less feels broken to users. Link to full blog in comments.

20

2 Comments