Senior Software Engineer, AI at Axon | Teaching AI Engineering at TheAIMerge
Braşov, România
34 K urmăritori
Peste 500 de contacte
34 K urmăritori
Peste 500 de contacte
Despre
I’ve been working in AI for close to a decade, starting well before today’s LLM-driven wave.
My work is grounded in understanding AI across layers — from hardware, AI models, and data to system design, reliability, and scale. I approach AI as an end-to-end engineering problem: not just modeling, but how models interact with data, infrastructure, and real-world constraints.
Alongside this, I write The AI Merge, a technical newsletter on AI foundations and systems engineering, focused on clear thinking and practical system design rather than hype.
I’m currently building an AI Systems course, centered on end-to-end system design, for engineers who want a deeper, systems-first understanding of modern AI.
🗞️ Read The AI Merge: https://read.theaimerge.com
Activitate
34 K urmăritori
-
Alex Razvant a distribuit aceastaPersonal update: I’ve joined Axon as a Senior Software Engineer, AI. Excited to join a great team and work on meaningful problems - building technology that helps protect life around the world. I’ve also been a bit quiet here lately while settling in and working on a few things behind the scenes. More soon.
-
Alex Razvant a distribuit aceastaAI inference and the merge between GPUs and DSAs GPUs currently handle the largest chunk of AI Compute for both model training and inference at scale. They won't go anywhere. However, demand for AI inference is growing and many companies are designing Domain Specific Architectures (DSAs), their own hardware. Some examples are: 1/ Google TPU (Tensor Processing Unit) 2/ Apple NPU (Neural Processing Unit) 3/ Groq LPU (Language Processing Unit) 4/ Cerebras 5/ Tenstorrent Blackhole 6/ AMD Instinct M 7/ SambaNova SN40L 8/ AWS Trainium/Inferentia 9/ NVIDIA Groq 3 LPX What's the problem DSAs are solving? > To minimize data movement. They optimize the memory hierarchy and the compute-bound vs memory-bound trade-off. In GPUs, a data point has to pass through VRAM, then SRAM, and Registers etc. Each component in this hierarchy increases in bandwidth but decreases in memory size. VRAM (Slowest)> Gigabytes. Caches (Faster) > Megabytes. Registers (Fastest) > Kilobytes. DSAs reorganize this hierarchy and reduce the time a data point spends in memory, to leave more for the compute part. That's why LLM inference is so fast on Groq LPUs for example. Follow for expert insights on AI Engineering & Systems.
-
Alex Razvant a distribuit aceastaA few local LLMs worth trying right now! This list mainly includes MoE models, all compatible with GGUF so you can test/run them locally with Ollama, LMStudio or with llama.cpp directly. 𝗟𝗼𝘄 𝗧𝗶𝗲𝗿 (𝘂𝗽 𝘁𝗼 𝟵𝗕, 𝟭𝟲𝗚𝗕 𝗩𝗥𝗔𝗠) > Qwen3.5-4B [3.10M+ downloads] Small, Strong for agentic workflows, reasoning, and tool use. > Google Gemma-4-4b-it [2.5M+ downloads] Efficient on low-resource devices, good for QA, text summarization and low-reasoning. > Microsoft Phi-4-mini-instruct [1.4M+ downloads] Nice fit for long-document summarization, QA, and retrieval-style work, supports 128K context. 𝗠𝗶𝗱 𝗧𝗶𝗲𝗿 (𝘂𝗽 𝘁𝗼 𝟰𝟬𝗕 𝗠𝗼𝗘, 𝟯𝟮𝗚𝗕 𝗩𝗥𝗔𝗠) > OpenAI GPT OSS 20B [6.5M+ downloads] Popular, good for agent workflows, with native support for function calling, web browsing. Mixture-of-Experts (MoE) architecture, 128K context. > Qwen3.5-35B-A3B [3.9M+ downloads] Native, multi-modal Mixture-of-Experts (MoE) architecture, 260k context size, support up to 1M. Only 3B parameters active, great for long-context, tool-calling/agentic workflows. > NVIDIA Nemotron Nano 30B-A3B [1.5M+ downloads] High on benchmarks, good at long-context instruct & agentic workloads, up to 1M tokens. Hybrid MoE Transformer (Mamba-2), 30B parameters total, 3.5B active, scales linearly in throughput once the input grows. 𝗟𝗮𝗿𝗴𝗲 𝗧𝗶𝗲𝗿 (𝟭𝟬𝟬𝗕+ 𝗠𝗼𝗘, 𝟯𝟮+ 𝗚𝗕 𝗩𝗥𝗔𝗠) > GPT-OSS-120b - https://buff.ly/0mIVh2g… > Nemotron 3 Super 120B-A12B - https://buff.ly/4EuHW2C… > Qwen3.5-122B-A10B- https://buff.ly/61IlGiv… Local AI is growing rapidly. SLMs keep getting better and better!
-
Alex Razvant a distribuit aceastaInstead of spending time learning about AI frameworks, learn the machine underneath them, compute. You do not need to become a CUDA or GPU specialist. But you do need a working mental model of how GPUs actually execute work. Once you have that, a lot of “advanced” topics stop looking advanced: 1/ Inference and Batching 2/ Quantization 3/ Kernel Fusion 4/ Memory Transfers, KV Cache 5/ Throughput, Host Metrics/Device Metrics Most problems during AI training and inference are not really model problems. They are often memory, scheduling, and data movement problems. This slide is one small piece of a larger set I’m building on GPUs and AI: compute, GPU architecture, model optimization, inference, training, and the system tradeoffs engineers actually run into. I’ll share the full deck through my newsletter when it’s ready. You can subscribe here: https://buff.ly/DnpHEXD
-
Alex Razvant a distribuit aceastaInstead of spending time learning about AI frameworks, learn the machine underneath them, compute. You do not need to become a CUDA or GPU specialist. But you do need a working mental model of how GPUs actually execute work. Once you have that, a lot of “advanced” topics stop looking advanced: 1/ Inference and Batching 2/ Quantization 3/ Kernel Fusion 4/ Memory Transfers, KV Cache 5/ Throughput, Host Metrics/Device Metrics Most problems during AI training and inference are not really model problems. They are often memory, scheduling, and data movement problems. This slide is one small piece of a larger set I’m building on GPUs and AI: compute, GPU architecture, model optimization, inference, training, and the system tradeoffs engineers actually run into. I’ll share the full deck through my newsletter when it’s ready. You can subscribe here: https://buff.ly/DnpHEXD
-
Alex Razvant a distribuit aceastaFound this nice, interactive guide on LLM Quantization If you're working with LLMs locally, or just want to understand how popular local LLM-inference solutions such as Jan, LMstudio, Ollama are capable of running your models - give it a read. In short: 1/ All these are using llama.cpp as the inference engine on the backend. 2/ Llama.cpp uses GGML Tensor Library 3/ GGML can load & parse GGUF model checkpoints * In a GGUF checkpoint, your model weights are stored in various quantizations formats (Q4, Q5_K, Q8_I_K, Q4_2, Q3_K_L etc). To understand what all those are 👇️ https://buff.ly/V0E4zXQ
-
Alex Razvant a distribuit aceastaOnline, I keep seeing many claim to push 10k Lines of Code a day, not reviewing anything and trusting everything. I think that’s marketing, or bad practice at least. The more code added to a codebase, the larger the surface area. Software isn't written once. It's designed to evolve. What works now, should be easily changed or improved in the future. The question you should ask: “Can we ship changes that are consistent and safe to release?” A few safeguards that actually worked for me: 1. Treat AI code as proposal, not truth. Don’t accept diffs you can’t explain. 2. Put a harness on the agent. (Cursor Rules / AgentsMD / ClaudeMD / Skills / Plugins) 3. Keep Agent Rules, Skills and Plugins small, scoped and enforceable. 4. Add guardrails/improve skills and plugins only after noticing Coding Agent failures. 5. Keep PRs reviewable. Minimal context + a 1-pager attached on the PR for big moves. 6. Reviews should be fast and should't focus on preference. Skip nitpicks. Focus on the entire workflow. 7. Pre-commit + CI parity. Catch drift before you waste a CI cycle. 8. Write a handoff note to your tomorrow-self so you catch-up faster on the next day. Add a Skill to your agent for that. For an Engineer, AI helps, but only when you stay in control of constraints. --- I explain complex topics in AI, AI Engineering and building AI Systems. For free. Join here: https://buff.ly/DnpHEXD
-
Alex Razvant a distribuit aceastaThe demand for AI inference, pushes for a merge between GPUs and DSAs. GPUs currently handle the largest chunk of AI Compute for both model training and inference at scale. They won't go anywhere. However, demand for AI inference is growing and many companies are designing Domain Specific Architectures (DSAs), their own hardware. Some examples are: 1/ Google TPU (Tensor Processing Unit) 2/ Apple NPU (Neural Processing Unit) 3/ Groq LPU (Language Processing Unit) 4/ Cerebras 5/ Tenstorrent Blackhole 6/ AMD Instinct M 7/ SambaNova SN40L 8/ AWS Trainium/Inferentia 9/ NVIDIA Groq 3 LPX What's the problem DSAs are solving? > To minimize data movement. They optimize the memory hierarchy and the compute-bound vs memory-bound trade-off. In GPUs, a data point has to pass through VRAM, then SRAM, and Registers etc. Each component in this hierarchy increases in bandwidth but decreases in memory size. VRAM > Gigabytes. Caches > Megabytes. Registers > Kilobytes. DSAs, amongst other things, propose to reorganize this hierarchy and reduce the time a data point spends in memory, to leave more for the compute part. That's why LLM inference is so fast on Groq LPUs for example. On the same note, the new NVIDIA's Vera Rubin line of GPUs + Groq LPUs, making the NVIDIA Groq 3 LPX Accelerator is super interesting in this regard. Seems like itt brings the best of both worlds. --- ✅️ I'm going to cover everything discussed above, and more (see first comment). Follow for expert insights on AI Engineering & Systems.
-
Alex Razvant a distribuit aceastaThis shiny new DGX Spark will be heading out to Spain this time 🇪🇸 Thanks to everyone who joined my DGX Spark giveaway for NVIDIA GTC 2026. The raffle is now officially closed, and the winner was picked at random, from 112 participants. I got in touch with him and had a fun chat on what cool AI projects and ideas he could now bring to life, something that wasn't possible on his CPU-only machine. We talked about OCR, describing images with VLMs and a few NemoClaw (from the DGX Playbooks) ideas that sounded interesting. Excited to see what gets built with it. I really appreciate everyone who took the time to register, attend sessions, and participate, GTC is always packed with valuable insights, and I hope you found at least a few sessions to learn something new or get inspired with new ideas. Thanks again to NVIDIA AI for making this possible, and to all of you for being part of it. P.S I'm working on a 30+ (11 left) slides presentation on GPUs for AI, packed with details, and will cover the Spark as well as various other AI insights. Subscribe to https://lnkd.in/dvjXy_BV to be notified for when I'll do the live session. 💚
-
Alex Razvant a reacționat la aceastaAlex Razvant a reacționat la aceastaI wrote this in a previous comment on Demitri Swan's post about paying closer attention to hardware architecture as an engineer (whatever you engineer, haha). But here is the thing. In a lot of programming discussion, we talk more in terms of language features — learn C/C++, Rust, Go etc. They are all fantastic. But there's something more fantastic than language features. Which is hardware architecture and a deeper understanding of the compiler — how it cleanly translates your higher-level expression to run on the target machine (many ninja programmers grab more power here, power you can never control at a high level of language abstraction, with compiler hints and flags). Take GPU kernel writing for example. The biggest bottleneck in many GPU kernels is data movement — global memory to shared memory, and now tensor memory (TMEM) in-between on Blackwell, down to registers. TMEM isn't incidental: 256 KB per SM on Blackwell Ultra, purpose-built for warp-synchronous reuse of intermediate results to cut off-chip traffic. NVIDIA designed the silicon around the exact problem most kernels are bottlenecked on. A lot of GPU content talks about memory coalescing and tiling strategies. But tiling without intelligent awareness of byte alignment and register physical capacity will cause register spillover. On Blackwell each thread caps at 255 registers against a 64K (65,536) 32-bit register file per SM — exceed that budget and you spill to local memory, losing the reuse you tiled for. Similarly, the TMA engine is a hardware feature deeply co-designed with the CUDA Pipeline API (CUDA 13+) to hide memory latency behind computation. Offload the global→shared copy from the SM's load/store units to TMA's DMA engine, double-buffer (ping-pong) the tiles, and you hide the ~800-cycle DRAM round trip while pushing global memory load efficiency toward 99%. And TMA awareness on Blackwell alone — plus instructing the compiler with const __restrict__ to route "mostly" read-only data through the read-only cache path — will improve your kernel efficiency by a far margin. In a typical lookup example, that one qualifier lifts SM Active % from 45% to 93% and drops DRAM throughput from 600 GB/s to 200 GB/s as loads hit cache instead of off-chip. #CUDA #GPUProgramming #Blackwell #HighPerformanceComputing #MLSystems
Experiență
-
-
-
-
-
Co Author
HandsOnLLMs
-
-
-
Upwork
-
-
Elektrobit
-
-
-
-
Freelance
NOISE Festival Brasov
-
-
Second Place 🥈
University Coding Challenge