Christos Kozyrakis

PhD student Zixi Li is pushing the boundaries of quantum hardware at Pritzker School of Molecular Engineering at the University of Chicago. Her work uses thin diamond films with atomic-scale defects as controllable qubits. This effort brings us closer to practical quantum technologies, from global quantum networks to real-world quantum devices. “I’m very proud that I’m a part of this second quantum revolution and part of this quantum hub,” she said. Learn more about PhD programs at UChicago PME: https://lnkd.in/gKd4xEhh

75

The Spring issue of IEEE Solid-State Circuits Magazine is out! https://lnkd.in/gxbjNtKw This issue is dedicated to Integrated Circuits for AI and ML applications, and was guest edited by Marian Verhelst and Charlotte Frenkel. It features four excellent articles that highlight core challenges and methodologies in AI hardware acceleration (see https://lnkd.in/gayteYmV for more detail). The feature articles are: 1) "Design and Optimization of Efficient Digital Machine Learning Accelerators" by Wei Tang, Sung-Gun Cho, Jie-Fang Zhang, and Zhengya Zhang examines AI acceleration through optimized computational kernels, such as matrix–matrix multiplication, leveraging architectures like SIMD and systolic arrays (10.1109/MSSC.2025.3549361). 2) "SRAM- and eDRAM-Based Compute-in-Memory Designs, Accelerators, and Evaluation Frameworks," by Yifan He, Xiaofeng Hu, Hongyang Jia, and Jae-sun Seo addresses in-memory computing (IMC) as a means to reduce data movement overhead, covering both analog and digital IMC approaches (https://lnkd.in/gHiukWKx). 3) "Open Source Heterogeneous SoCs for Artificial Intelligence," by Francesco Conti, Angelo Garofalo, Davide Rossi, Giuseppe Tagliavini, and Luca Benini explores the integration of heterogeneous systems on chip, demonstrating how open source platforms such as PULP support scalable AI hardware development (https://lnkd.in/gAQgK-8d). 4) " Sparsity-Aware Hardware," by Man Shi, Adrian Kneip, Nicolas Chauvaux, Jiacong Sun, Charlotte Frenkel, and Marian Verhelst discusses sparsity-aware hardware design, illustrating how structured and unstructured sparsity can be leveraged for significant performance improvements using an adapted roofline model (https://lnkd.in/gv8eTyvd). The issue also features regular columns: 1. Behzad Razavi’s “The Analog Mind” column (https://lnkd.in/gxKyqZr7) continues the series on CMOS inverter applications 2. Chris Mangelsdorf’s “Shop Talk: What You Didn’t Learn in School” column picks up where the previous issue left off, continuing the discussion on Miller compensation in "Miller’s Wrong Half-Plane Zero" (https://lnkd.in/gngTrywV). This time, the focus is on the right half-plane zero and its potential to degrade stability if not properly addressed. 3. Farhana S.’s “Circuits From a Systems Perspective” traces the historical evolution of wireless charging in everyday devices (https://lnkd.in/gnVhqUyF). Lot's of interesting articles to read! Enjoy! #ieeesscc #analog #ieee #ai #ml #solidstatecircuits #hardwareacceleration #semiconductors #inmemorycomputing

260

3 Comments

Introducing Generative Adversarial Distillation (GAD): a novel GAN-style formulation and framework that facilitates both on-policy and black-box distillation of large language models (LLMs). GAD is the first technique to enable block-box on-policy distillation from proprietary teachers where internal logits or parameters are inaccessible, or distillation between teacher and student LLMs with incompatible vocabularies. GAD expands our prior work on white-box on-policy distillation (i.e., MiniLLM), pioneering block-box on-policy distillation for LLM training. Specifically, GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM’s, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation.  In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation. Our team has been conducting fundamental research in knowledge distillation with wide adoptions across the industry. -         MiniLM: We introduced multi-head attention distillation, establishing the most effective distillation method for BERT-style models. The open-source MiniLM models (e.g., 6x384) have become the most widely utilized small encoder models on the Hugging Face. -         MiniLLM: Our proposed Reverse KLD is recognized as one of the most effective, de facto on-policy distillation approaches for modern LLM training, which has been widely used by Thinking Machines, Gemma, and many other teams and models. -         BitDistill: We proposed BitNet Distillation to finetune off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (ternary weights {-1, 0, 1}), achieving performance parity with the full-precision counterparts on specific downstream tasks. -         GAD: The development of Generative Adversarial Distillation (GAD) now allows for black-box on-policy distillation, overcoming two major prior limitations: (1) Distillation from proprietary teachers where internal logits or parameters are inaccessible; (2) Distillation between teacher and student LLMs with incompatible vocabularies. https://lnkd.in/gMaP2c7w

125

2 Comments

KKL Observer Synthesis for Nonlinear Systems via Physics-Informed Learning by M. Umar B. Niazi, John Cao, Matthieu Barreau and Karl Henrik Johansson Abstract This paper proposes a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for autonomous nonlinear systems. The design of a KKL observer involves finding an injective map that transforms the system state into a higher-dimensional observer state, whose dynamics is linear and stable. The observer's state is then mapped back to the original system coordinates via the inverse map to obtain the state estimate. However, finding this transformation and its inverse is quite challenging. We propose to sequentially approximate these maps by neural networks that are trained using physics-informed learning. We generate synthetic data for training by numerically solving the system and observer dynamics. Theoretical guarantees for the robustness of state estimation against approximation error and system uncertainties are provided. Additionally, a systematic method for optimizing observer performance through parameter selection is presented. The effectiveness of the proposed approach is demonstrated through numerical simulations on benchmark examples and its application to sensor fault detection and isolation in a network of Kuramoto oscillators using learned KKL observers.

20

1 Comment

New paper out! We design a fault-tolerant quantum memory distributed over multiple modules and we show that distributed quantum LDPC codes can maintain their performance in a distributed setting while using solely one simple connector between modules: a cyclic shift. https://lnkd.in/e3yDYNBv Thank you to my amazing teammates Edwin Tham, Min Ye, Ilia K. and John Gamble.

216

5 Comments

🚀 Impressive ISCA’25 paper dissecting DeepSeek’s 🧠 compute stack: https://lnkd.in/e-9h493N 💡 FP8 precision, MLA, MoEs, Multi-plane Fat Trees w/ endpoint routing via NCCL’s PXN! 📐 The topology? Equivalent to a 1D HammingMesh—clean fit for ~2k nodes. 2D HammingMesh could scale much further (shameless plug): https://lnkd.in/eK8H4QRQ #HPC #AI #TopologyDesign #ISCA25

154

1 Comment

Oskar Painter provides an inside look at Amazon's effort to advance fault-tolerant quantum computing using "cat qubits" at the Center for Quantum Computing at Caltech. My comment (12:25 in the video): “The approach that Amazon is taking now at the Center for Quantum Computing is a bit different than what other companies are pursuing. It’s promising, but whether it’ll pan out, we’re not sure. A company like Amazon can afford to take that type of risk. I’m impressed by the extent to which the Amazon leadership appreciates that this is a long-term enterprise.” https://lnkd.in/echTdb3h

180

4 Comments

Excited that our paper "Solving sparse finite element problems on neuromorphic hardware", led by Brad Theilman, was just published in Nature Machine Intelligence! There is a lot to say about this story. But long story short, we show that today's ultra low-power neuromorphic systems can be used to solve sparse linear systems, one of the major workhorses of scientific computing. We show that our method, dubbed NeuroFEM, is numerically comparable to conventional linear solvers and exhibits near-ideal strong and weak scaling. Further, unlike many other neuromorphic algorithms, its use does not require users master spiking neural networks; if you can define a mesh and solve Ax=b using a standard linear solver, you can use NeuroFEM. Importantly, the NeuroFEM algorithm is truly brain inspired, building on dynamical models of motor cortex. This provides the algorithm a natural brain-like sparsity, a local topology, and distributed asynchronous dynamics that combined naturally maximize the advantages offered by neuromorphic systems. This is just a first step, and we are optimistic that this may open the door to more widespread and accessible applications of neuromorphic hardware. Iterative solvers such as FEM underlie many numerical methods, and this may help inspire new strategies for energy efficient AI and bringing scientific computing to the edge. Thanks to the DOE Office of Science ASCR / BES Codesign in Microelectronics project COINFLIPS for support in the codesign of advanced neural algorithms such as this! Open access here: https://lnkd.in/gTcd5DTX

218

4 Comments

Our work advancing quantum algorithms for nonlinear PDEs is now published in npj Quantum Information. This is an important potential application of quantum computers in simulating classical systems, but the nonlinear case is highly challenging for quantum algorithms. https://lnkd.in/gwURTaVA

173

5 Comments

Over a decade ago, Duke ECE faculty Yiran Chen tackled a challenge that seemed ahead of its time: training neural networks directly on hardware. His paper published in 2014 laid the groundwork for modern machine learning, enabling AI to process data faster and more efficiently. Last year, Chen and his co-authors won a 10-Year Retrospective Most Influential Paper Award from the Asia and South Pacific Design Automation Conference (ASP-DAC). 💡 🔗 https://lnkd.in/erQ5RTvH #DukeEngineering #AI #MachineLearning

458

25 Comments

Automated unit testing, including test oracles, using multiple, specialized, and collaborating LLM agents This is a very exciting piece of new work led by Qinghua Xu, a research fellow in my group at the Lero research centre in Ireland, in collaboration with Huawei. "Hallucination to Consensus: Multi-Agent LLMs for End-to-End JUnit Test Generation", Qinghua Xu, Guancheng Wang, Lionel Briand, Kui Liu https://lnkd.in/enu5adY3 We propose CANDOR, a novel end-to-end, prompt engineering-based LLM framework for automated unit test generation in Java. CANDOR orchestrates multiple specialized LLM agents to collaboratively generate complete JUnit tests, including both high-quality test prefixes and accurate oracles.

60

1 Comment