Hazem Abdelazim’s Post

Exciting news from PyTorch Conference 2025! The latest launches are set to redefine AI development: • Monarch: A single-controller orchestration engine for streamlined distributed workflows. • Forge: Native PyTorch reinforcement learning for seamless RL integration. • TorchComms: A modern comms API to supercharge PyTorch communication layers. • Helion: A higher-level kernel authoring DSL for simplified GPU programming. • ExecuTorch GA: General availability of PyTorch at the Edge, powering efficient on-device AI. These innovations promise enhanced scalability, performance, and ease of use—marking a big leap for the PyTorch ecosystem! #PyTorch #AI2025 #TechLaunch

For years, individuals have relied on "fine-tuning" as the primary approach to improve the performance of large language models. This entails providing the model with fresh data, modifying several weights, and listening to the GPU fans roar, all in anticipation of enhancing its performance next time. However, Stanford recently released a paper that essentially claims "There's no need for retraining whatsoever." The idea is referred to as Agentic Context Engineering, or just ACE, and it’s ingenious. Rather than modifying model weights, ACE enhances the context surrounding the model ,referring to it as the prompt. Thus, rather than retraining, the model just modifies its own playbook as it advances

Very impressive blog, the non-determinism seen in LLM online inference is not a flaw in the inference engine, but stems from how GPU kernels are implemented for performance optimization. It's impressive that such simple experiments resulted in such significant findings. I've noticed that these blog posts by thinkingmachines.ai are of very high quality. https://lnkd.in/gdqe-3NB

More compute doesn't always mean smarter AI. We're seeing models hit performance plateaus because they often don't explore enough different ways to solve a problem during training. A new paper on a framework called DeepSearch offers a fascinating fix: instead of just learning from one answer at a time, it teaches the model to explore a whole tree of possibilities while it's still learning. The result is the real eye-opener. It not only set a new state-of-the-art for models its size but did it using almost 6 times less compute. It’s a powerful reminder that smarter algorithms can beat brute-force scaling. This is a hopeful sign that we can build better AI by being more creative, not just by using more power. A cool read for anyone in the AI space. You can find the model on Hugging Face: [https://lnkd.in/gQp87fdA]. #AI #MachineLearning #LLM #Efficiency

🚀 In the 34th AI Talks, we dived into the challenges of deploying Large Language Models (LLMs) with Farbod Bijary. From GPU optimizations and parallelism to navigating Persian LLM leaderboards, the session shared practical insights on making LLMs faster, smarter, and ready for real-world use. 💡✨ Key takeaways: ⚡️ Deploying LLMs – CPU vs GPU trade-offs, costs, and setup choices 📊 Persian LLM benchmarks – when to rely on them (and when not to) 🏎️ Speed & optimization – quantization, Torch Compile, 5D parallelism 🎯 Practical deployment – balancing quality, speed, and user needs

This week I had the opportunity to make a presentation on the challenges of deploying LLMs in the 34th AI Talks event. I demonstrated how the performance of LLMs is evaluated on the Persian language and how the latency and throughput of these models can be optimized. Glad to have connected with this fun and technical community. 🫡 P.S. Here's the link of my presentation if you're interested: https://lnkd.in/ddarP33S

GGML, a C library - machine learning Tensor Library. GGUF and Quantization for Edge LLM model Inference. The tutorial provides an extensive overview of GGML, a C library specifically designed for the efficient inference of large language models (LLMs) on consumer-grade hardware, particularly through CPU optimization and broad hardware compatibility. A core feature of GGML is its support for quantization, a process that reduces model precision to significantly decrease size and memory consumption, thereby enabling "inference at the edge." The sources explain that GGML operates using tensors and a computational graph for defining and executing operations, and they also introduce GGUF, a successor binary format optimized for storing models for GGML-based executors. Finally, the texts highlight the minimalism, portability, and community-driven development of GGML, demonstrating its use through low-level C code examples for matrix multiplication and neural network inference. https://lnkd.in/geDYDxZC

A 300M-parameter language model just learned to predict GPU latency, memory usage, and model accuracy by reading code as plain text. No feature engineering. No graph encoders. Just pure text-to-number prediction with 0.93 correlation on memory benchmarks. Performance optimization is getting a major upgrade. Read how: https://lnkd.in/gM24AWRb #MachineLearning #ArtificialIntelligence #DeepLearning #MLOps #GPUComputing #CodeOptimization #NeuralNetworks #PerformanceEngineering #TechInnovation #AIResearch

Libraries: Eigen, Boost, or custom GPU kernels support quaternion math. For large systems, storage scales linearly with quaternions per site but computations scale slightly worse due to non-commutativity. When simulating quaternionic Schrödinger equations, careful attention must be paid to order of multiplication in updates. Drawing: https://lnkd.in/dhdHSKqn

Ten years ago, you’d need an entire research team and large budget to build an AI that can recognize different types of cats, today, we have AI systems that can generate new realities within seconds. it's really exciting what one can do with just a few lines of code For example, I fine-tuned the small ResNet18 image model on CPU and achieved 0.000013% validation loss(tiny data set) with 100% prediction accuracy so far. The core model was trained in a Jupyter Notebook using fastai, pickled and deployed to huggingface space with gradio, all in under 60 lines of code. #fastai #finetuning #AI