SemiAnalysis (@SemiAnalysis

Samtech displayed their Co-Packaged Copper (CPC) set-up in the CPX form factor at Taiwan OCP (See the open CPX MSA here: opencpxmsa.org). Bandwidth exits the 6.4T pluggable optical engine via a copper connector interface. There are 128 connector pins per module

Amazing work by NVIDIA & Google on implementing auto-activation out of the box for Google's NCCL Plugin for ConnectX-7/8 NCCL. This was implemented due to feedback from SemiAnalysis a couple of quarters ago to massively improve the out-of-the-box experience on the GCP platform!

Cerebras is great, & the people at Cerebras are great & lovely! But since NVIDIA showed off their LPX's amazing perf at Hot Chips & OpenAI Jalapeno performance was released, Cerebras folks have been massively cope-tweeting on Twitter. They should focus on continuously improving

Huawei showcased the Atlas 950 SuperPoD servers for the first time in WAIC after first announcing it at MWC in March. An Atlas 950 SuperPoD consists of 1,024 NPUs, which consists of 16x Atlas 950 racks with each rack having 64 Ascend 950 NPUs. Each 950 SuperPoD therefore boasts 1

NVIDIA LPU supports 3 types of disaggregated inferencing: 1. Rubin Prefill + LPU Decode for the fastest interactivity 2. Rubin Prefill + Rubin Decode Attention + LPU Decode FFN for the middle of the curve 3. Rubin Prefill + Rubin Decode Verification + LPU Drafter for the

49