Python’s GIL in 2025: What Changes, What Doesn’t (3.14) ⚙️🐍 • GIL recap: In CPython, the Global Interpreter Lock forces at most one running thread per process, so CPU-bound multi-threading doesn’t scale across cores. I/O-bound threads are fine; CPU-bound work benefits from multiprocessing. • What’s new in 3.14: Optional no-GIL builds aim to remove that bottleneck. Expect wins on CPU-bound workloads with true multithreading—if deps and extensions play nicely. • Engineering guidance: • For I/O: threading is still great. • For CPU: today → multiprocessing/vectorized libs; explore no-GIL builds as your deps gain support. • Profile first: many real-world pipelines are I/O-bound or C-accelerated already. • Prod checklist: Watch extension compatibility, thread-safety, and memory contention; measure throughput and P95 latency before flipping defaults. Hashtags: #dailydoseofds #datascience #machinelearning #MLOps #Python #GIL #Concurrency #Multithreading #Multiprocessing
Python 3.14: No-GIL builds for CPU-bound workloads
More Relevant Posts
-
Python launched its latest 3.14 version — and it brings two major updates that change everything! 🚀 💡 1️⃣ No more GIL (Global Interpreter Lock) Earlier, Python could only run one thread at a time, even on multi-core CPUs. With Python 3.14, GIL is finally optional, meaning: ✅ True multi-threading ✅ Faster performance for AI, ML, and data-heavy tasks ✅ Better use of all CPU cores 🧵 2️⃣ T-Strings (Template Strings) A new way to handle strings! t"Hello {name}" doesn’t just print text — it keeps the structure of your template, making string handling safer, cleaner, and more powerful for developers. 👉 Does this change make your coding life easier? Let me know in the comments below 👇 #python314 #AIML #pythonupdate #linkedin
To view or add a comment, sign in
-
-
Profiling a HPC code that makes use of concurrency or parallelism using languages and libraries like C++, MPI, OpenMP, CUDA and Python help identify the performance bottlenecks. NVIDIA's Nsight system simplifies this task using tools like NVTX and CUDA Profiling API. Besides these, Pytorch comes with its own profiling modules. I have put these in one place by preparing a set of slides for a training that I conduct. These slides also contain a brief reference to the general workflow for debugging a code using GDB, CUDA-GDB and PDB. The slides can be downloaded from https://lnkd.in/gCUWp5cS
To view or add a comment, sign in
-
-
#Day36 of my fourth #100DaysOfCode Just leveled up my Python game with these 5 game-changing scripts! 1. Object Detection: Real-time object detection using YOLOv3 - because machines that can see are the future! 2. OCR: Extracting text from images with Tesseract-OCR - turning images into editable text in seconds! 3. ORB Algorithm: Feature matching between two images with insane precision - because details matter! 4. File Organizer: Organizing files into folders based on extensions - because who doesn't love a tidy digital life? 5. CPU Monitor: Alerting me when CPU usage exceeds a threshold - because system crashes are so last season! Code is live on my GitHub! [https://lnkd.in/gsTcpcAw] #Python #MachineLearning #ComputerVision #ProductivityHacks #Automation #CodingLife #Scripting #DevCommunity #TechInnovation #ArtificialIntelligence #ObjectDetection #OCR #FeatureMatching #FileOrganization #CPUMonitor #GitHub #OpenSource
To view or add a comment, sign in
-
𝗣𝘆𝘁𝗵𝗼𝗻 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝘁𝗵𝗲 𝗚𝗜𝗟, 𝘄𝗵𝘆 𝗲𝘃𝗲𝗿𝘆𝗼𝗻𝗲’𝘀 𝘁𝗮𝗹𝗸𝗶𝗻𝗴 𝗮𝗯𝗼𝘂𝘁 𝗶𝘁 If, like me, you had heard of the “GIL” for years but never quite understood why people complained about it… here’s the simple version 👇 Until now, 𝗣𝘆𝘁𝗵𝗼𝗻 𝗵𝗮𝗱 𝘀𝗼𝗺𝗲𝘁𝗵𝗶𝗻𝗴 𝗰𝗮𝗹𝗹𝗲𝗱 𝘁𝗵𝗲 𝗚𝗹𝗼𝗯𝗮𝗹 𝗜𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗲𝗿 𝗟𝗼𝗰𝗸 (𝗚𝗜𝗟). It’s basically a mutex that ensures that 𝘰𝘯𝘭𝘺 𝘰𝘯𝘦 𝘵𝘩𝘳𝘦𝘢𝘥 𝘦𝘹𝘦𝘤𝘶𝘵𝘦𝘴 𝘗𝘺𝘵𝘩𝘰𝘯 𝘣𝘺𝘵𝘦𝘤𝘰𝘥𝘦 𝘢𝘵 𝘢 𝘵𝘪𝘮𝘦, even on multi-core CPUs. So when you were running “multi-threaded” code in Python, you actually weren’t using multiple cores efficiently. You could have 𝘮𝘶𝘭𝘵𝘪𝘱𝘭𝘦 𝘵𝘩𝘳𝘦𝘢𝘥𝘴, yes, but 𝘰𝘯𝘦 𝘢𝘤𝘵𝘪𝘷𝘦 𝘢𝘵 𝘢 𝘵𝘪𝘮𝘦. That’s why people went for multiprocessing or async I/O to achieve real parallelism. 🐍 𝗣𝘆𝘁𝗵𝗼𝗻 𝟯.𝟭𝟰 𝗶𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗿𝗲𝗺𝗼𝘃𝗶𝗻𝗴 𝗶𝘁. That means: 🟢 True parallelism inside a single Python process 🟢 Major performance gains for data-intensive workloads 🟢 Simpler concurrency models — less boilerplate, fewer workarounds The technical challenge behind this removal is huge: the GIL was deeply woven into CPython internals. The new design introduces per-interpreter memory management, atomic reference counting, and fine-grained locks to make it work safely. 💡𝗜𝗻 𝘀𝗵𝗼𝗿𝘁: 𝗣𝘆𝘁𝗵𝗼𝗻 𝗶𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗰𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝘂𝗽 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗶𝘁’𝘀 𝗯𝗲𝗲𝗻 𝗿𝘂𝗻𝗻𝗶𝗻𝗴 𝗼𝗻 𝗳𝗼𝗿 𝗱𝗲𝗰𝗮𝗱𝗲𝘀. For data scientists, ML engineers, and backend devs, this could mean faster model training, snappier APIs, and simpler concurrency code. What do you think, will you upgrade your python version to taste true parallelism? #Python #SoftwareEngineering #DataEngineering #Performance #Concurrency
To view or add a comment, sign in
-
-
Python 3.14 finally brings a no-GIL version, which means threads can now run in true parallel and use all CPU cores. I feel this could really help frameworks like Frappe, faster background jobs, better use of CPU, and less memory load since we may not need too many worker processes. But I’m curious to know what others think will this actually make a big difference in real-world Frappe performance? #Python #NoGIL #Frappe #Performance
To view or add a comment, sign in
-
Here’s my short take after finishing 𝗡𝗩𝗜𝗗𝗜𝗔’𝘀 “𝗙𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹𝘀 𝗼𝗳 𝗔𝗰𝗰𝗲𝗹𝗲𝗿𝗮𝘁𝗲𝗱 𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗖𝗨𝗗𝗔 𝗣𝘆𝘁𝗵𝗼𝗻”: 𝐖𝐡𝐚𝐭 𝐈 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐥𝐞𝐚𝐫𝐧𝐞𝐝 * How to use 𝐍𝐮𝐦𝐛𝐚 (@cuda.jit) to write CUDA kernels in Python * The core CUDA execution model (grids, blocks, threads) and how to choose a launch configuration that maps well to data-parallel work * Moving data between NumPy on the host and device arrays, then measuring the speedup on a real linear-algebra style workload * Effective use of the memory subsystem, shared memory, basics of coalescing 𝐕𝐞𝐫𝐝𝐢𝐜𝐭 If you’re a Python user who wants real, hands-on GPU speedups without jumping straight into C++, this course is absolutely worth the money. It’s short (about 8 hours) with focused, step-by-step labs that make the concepts stick. #nvidia #cuda #GPU #numba #gpgpu #highperformancecomputing #deeplearning #parallelprogramming #python
To view or add a comment, sign in
-
-
Day:79/100 Recursion of Palindrome section ✅️ #100daysofcodingchallenge Recursion: ✔️Function calls itself: A function invokes itself repeatedly until a base case is reached. ✔️Base case: A condition that stops the recursion. ✔️Recursive case: The function calls itself with a smaller input or a modified version of the original input. ✔️Stack overflow: A potential risk if recursion is too deep. ✔️Memoization: Caching results to avoid redundant calculations. ✔️Tail recursion: A special case where the recursive call is the last operation. ✔️Base case: Empty or single-character strings are palindromes. ✔️ Recursive case: Compare first and last characters, then recurse on the substring. ✔️Symmetry: Palindromes have symmetry around their center. Benefits: 🔸️ Elegant code: Recursion can make the code more concise and readable. 🔸️ Easy to implement: Recursive palindrome checks can be simple to understand and implement. #Nxtwave #Intensive #100dayscoding #Python #Tech #coding #Programming #TechSkills #CareerDevelopment #DataLiteracy #BusinessIntelligence
To view or add a comment, sign in
-
Hello Everyone 👋 , 🤔 Is your Python code running slower than you expected? That’s because Python runs on a single core by default. But when your task is CPU-bound (like heavy calculations or simulations), you can unleash your system’s full potential using #ProcessPoolExecutor from concurrent.futures. This method distributes tasks across multiple CPU cores, enabling true parallel processing — and cutting execution time dramatically. ✅ Ideal for: CPU-heavy operations ❌ Avoid for: I/O-bound work (use ThreadPoolExecutor instead) 🔹 Just wrap your loop or function inside ProcessPoolExecutor() and experience the performance boost. Simple. Powerful. Efficient. 💪 Checkout attached docs for example. #contact: navinkpr2000@gmail.com #Python #ProcessPoolExecutor #Concurrency #ParallelComputing #Multiprocessing #Performance #Optimization #CodeTips #PythonDevelopers #Programming #AI #MachineLearning #DataScience #TechCommunity #Developers #crewxdev
To view or add a comment, sign in
-
🚀 Big news for Python developers! The Global Interpreter Lock (GIL) - the single biggest limitation for Python’s performance - is finally optional. Starting with Python 3.14, there’s experimental support for a “free-threaded” mode, which removes the GIL and allows true parallelism across CPU cores. 🧵 What this means: 💡 CPU-heavy and AI/ML workloads can finally leverage multiple cores in parallel. 💡 You’ll be able to run multi-threaded Python code that actually scales with your hardware. 💡 Perfect for numerical computing, data pipelines, and model inference on multicore CPUs. However: ⚙️ It’s still experimental and must be explicitly enabled during compilation. ⚙️ Not all libraries are thread-safe yet - migration will take time. Still, this is a historic step in Python’s evolution. From “one core only” to true multicore concurrency 🔥 #Python #AI #MachineLearning #DataScience #OpenSource #Performance #Developers
To view or add a comment, sign in
Extension compatibility really does feel like the make-or-break factor here. Most teams will likely need those careful profiling runs and gradual rollout approaches to see if this actually works in their production environments.