Developer Productivity Metrics

You're a #CTO. Your board asks: "What's our ROI on AI coding tools?" Your answer: "40% of our code is AI-generated!" They respond: "So what? Are we shipping faster? Are customers happier?" Most CTOs are measuring AI impact completely wrong. Here's what some are tracking: - Percentage of AI-generated code - Developer hours saved per week - Lines of code produced - AI tool adoption rates These metrics are like measuring how fast your assembly line workers attach parts while ignoring whether your cars actually start. Here's what you SHOULD measure instead: 1. Delivered business value 2. Customer cycle time 3. Development throughput 4. Quality and reliability 5. Total cost of delivery (not just development) 6. Team satisfaction Software development isn't a typing competition—it's a complex system. If AI makes your developers 30% faster but your deployment takes 2 weeks and QA adds another week, your customer delivery improves by maybe 7%. You've speed up the wrong part. The solution: A/B test your teams. Give half your teams AI tools, measure business outcomes over 2-3 release cycles. Track what customers actually experience, not how much developers produce. Companies that measure business impact from AI will pull ahead. Those measuring vanity metrics will wonder why their expensive tools aren't moving the needle. Stop measuring how much code AI generates. Start measuring how much faster you deliver value to customers. What are you actually measuring? And is it moving your business forward? -> Follow me for more about building great tech organizations at scale. More insights in my book "All Hands on Tech"

Has Amazon cracked the code on developer productivity with its cost to serve software (CTS-SW) metric? Amazon applied its well-known "working backwards" methodology to developer productivity. "Working backwards" in this case starting with the outcome: concrete returns for the business. This is measured by looking at the rate of customer-facing changes delivered by developers, i.e. "what the team deems valuable enough to review, merge, deploy, and support for customers", in the words of the blog post by Jim Haughwout https://lnkd.in/eqvW5wbi . This metric is different from other measures of developer productivity which look only at velocity or time saved. Instead, "CTS-SW directly links investments in the developer experience to those outcomes by assessing how frequently we deliver new or better experiences. Some organizations fall into the anti-pattern of calculating minutes saved to measure value, but that approach isn’t customer-centered and doesn’t prove value creation." This aligns with Gartner's own research on developer productivity. In our 2024 Software Engineering survey, we asked what productivity metric organizations are using to measure their developers. We also asked about a basket of ten success metrics, including software usability, retention of top performers, and meeting security standards. This allowed us to find out which productivity metric was associated most with success. What we found in our survey was that *rate of customer-facing changes* is the metric most associated with success. Some other productivity metrics were actually *negative associated* with success. But *rate of customer-facing changes* is what organizations should focus on. Sadly, our survey found that few organizations (just 22%) use this metric. I presented this data at our #GartnerApps summit [and the next summit is coming up in September: https://lnkd.in/ey2kpc2 ] Every metrics gets gamed. So I always recommend "gaming the gaming". A developer might game the CTS-SW metric by focusing more on customer-facing changes. But... this is actually a good thing. You're gaming the gaming. We will be watching closely how this metric gets adopted alongside DORA, SPACE, and other metrics in the industry.

Probably the simplest most-effective way to improve productivity is to reduce your work in progress (things you work on simultaneously) to 1. Think about a situation where you must work with a "platform team." Your team is bopping along until it comes across something it needs to do that the platform can't handle. It then stops work and hands off to the platform team. Rather than being idle while it waits, the first team now starts working on a second thing until it needs a database change, which it hands off to the database team. Not wanting to be idle, it starts working on a third thing. Weinberg points out that every "thing" you work on reduces productivity by about 20%. So, if you have three 5-day tasks. Working on two of them at once adds 20% to each task, so it will take 12 days to do 10 days of work. Add a third task and we're adding 2 days to each task, so it now will take 21 days to do 15 days of work. This isn't even considering what happens if the other team gets it wrong and you need to resubmit the request or the fact that it now takes up to four times longer (21 days rather than 5) to get something useful into your customer's hands. So, to work on only one thing at a time, we need to eliminate the dependencies. Our single product team needs to be able to make platform and database changes (safe ones, at least, to avoid collisions with other teams). They need to align with the other teams when they make those changes so that they don't break anything, but I find that an occasional chapter/guild meeting to deal with consistency issues takes way less time than the time you lose to WIP>1.

🚀 𝐓𝐡𝐞 2024 𝐒𝐭𝐚𝐭𝐞 𝐨𝐟 𝐃𝐞𝐯𝐎𝐩𝐬 𝐑𝐞𝐩𝐨𝐫𝐭 𝐢𝐬 𝐎𝐮𝐭! 🚀 The latest 𝐃𝐎𝐑𝐀 (DevOps Research and Assessment) report has dropped, packed with insights on how 𝐀𝐈 and 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 are reshaping 𝐃𝐞𝐯𝐎𝐩𝐬 and 𝐬𝐨𝐟𝐭𝐰𝐚𝐫𝐞 𝐝𝐞𝐥𝐢𝐯𝐞𝐫𝐲 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞. From faster deployment times to the unexpected effects of AI. Here are the 𝐤𝐞𝐲 𝐡𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: 📊 𝐓𝐡𝐞 𝐅𝐨𝐮𝐫 𝐊𝐞𝐲 𝐌𝐞𝐭𝐫𝐢𝐜𝐬: A Blueprint for High Performance DORA's foundation rests on four metrics that define software delivery excellence: 𝐥𝐞𝐚𝐝 𝐭𝐢𝐦𝐞 𝐟𝐨𝐫 𝐜𝐡𝐚𝐧𝐠𝐞, 𝐝𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭 𝐟𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐲, 𝐜𝐡𝐚𝐧𝐠𝐞 𝐟𝐚𝐢𝐥 𝐫𝐚𝐭𝐞, and 𝐫𝐞𝐜𝐨𝐯𝐞𝐫𝐲 𝐭𝐢𝐦𝐞. This year's report shows that teams who excel across these metrics achieve elite performance, 𝐫𝐞𝐠𝐚𝐫𝐝𝐥𝐞𝐬𝐬 𝐨𝐟 𝐢𝐧𝐝𝐮𝐬𝐭𝐫𝐲. 👉 It’s proof that high performance depends not on sector but on 𝐬𝐦𝐚𝐫𝐭, 𝐞𝐟𝐟𝐞𝐜𝐭𝐢𝐯𝐞 𝐃𝐞𝐯𝐎𝐩𝐬 𝐩𝐫𝐚𝐜𝐭𝐢𝐜𝐞𝐬. 🤖 𝐀𝐈 𝐀𝐝𝐨𝐩𝐭𝐢𝐨𝐧 = 𝐇𝐢𝐠𝐡𝐞𝐫 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲... with a Twist 𝐀𝐈 has gone 𝐦𝐚𝐢𝐧𝐬𝐭𝐫𝐞𝐚𝐦, with 81% of organizations now integrating it into 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭 workflows. DevOps professionals report big 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲 𝐛𝐨𝐨𝐬𝐭𝐬, but there’s a paradox: AI speeds up "𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞" work but leaves more time for the 𝐫𝐞𝐩𝐞𝐭𝐢𝐭𝐢𝐯𝐞 𝐭𝐚𝐬𝐤𝐬 we’re all too familiar with. 👉 Meaning AI makes the 𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞 work 𝐪𝐮𝐢𝐜𝐤𝐞𝐫, but it 𝐝𝐨𝐞𝐬𝐧’𝐭 𝐞𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐞 everything we 𝐝𝐨𝐧’𝐭 𝐰𝐚𝐧𝐭 𝐭𝐨 𝐝𝐨 😢! 🛠️ 𝐏𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠: Powers Developer Independence ���𝐧𝐭𝐞𝐫𝐧𝐚𝐥 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐫 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦𝐬 are transforming team workflows by enabling self-service. Teams with strong platforms report a 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲 increase of 8% and 𝐭𝐞𝐚𝐦 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 boost of 10%! However, with more layers in the pipeline, overall delivery speed can sometimes slow. 👉 For the best results, 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 should focus on 𝐮𝐬𝐞𝐫-𝐜𝐞𝐧𝐭𝐞𝐫𝐞𝐝 design, developer 𝐢𝐧𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐜𝐞, and a 𝐩𝐫𝐨𝐝𝐮𝐜𝐭-𝐨𝐫𝐢𝐞𝐧𝐭𝐞𝐝 approach. ❤️🔥 𝐓𝐡𝐞 𝐑𝐨𝐥𝐞 𝐨𝐟 𝐋𝐞𝐚𝐝𝐞𝐫𝐬𝐡𝐢𝐩 & 𝐒𝐭𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐥𝐞𝐚𝐝𝐞𝐫𝐬𝐡𝐢𝐩 that prioritizes stability and 𝐮𝐬𝐞𝐫-𝐜𝐞𝐧𝐭𝐫𝐢𝐜𝐢𝐭𝐲 is a game-changer. Teams with consistent 𝐩𝐫𝐢𝐨𝐫𝐢𝐭𝐢𝐞𝐬 and 𝐮𝐬𝐞𝐫-𝐟𝐨𝐜𝐮𝐬𝐞𝐝 goals produce better software, experience less burnout, and enjoy higher job satisfaction. 👉 Set the 𝐠𝐨𝐚𝐥 for your organization and teams to be just a 𝐥𝐢𝐭𝐭𝐥𝐞 𝐛𝐞𝐭𝐭𝐞𝐫 than yesterday. 𝐓𝐡𝐞 𝐌𝐚𝐢𝐧 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲: 💡 AI and platform engineering are powerful, but they’re not magic solutions. Elite performance is about 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭, 𝐞𝐱𝐩𝐞𝐫𝐢𝐦𝐞𝐧𝐭𝐢𝐧𝐠, and staying 𝐮𝐬𝐞𝐫-𝐟𝐨𝐜𝐮𝐬𝐞𝐝. 📎 Link to the full report is in the comments! #DevOps #AI #PlatformEngineering #StateOfDevOps #DORA #Leadership

As a client project manager, I consistently found that offshore software development teams from major providers like Infosys, Accenture, IBM, and others delivered software that failed 1/3rd of our UAT tests after the provider's independent dedicated QA teams passed it. And when we got a fix back, it failed at the same rate, meaning some features cycled through Dev/QA/UAT ten times before they worked. I got to know some of the onshore technical leaders from these companies well enough for them to tell me confidentially that we were getting such poor quality because the offshore teams were full of junior developers who didn't know what they were doing and didn't use any modern software engineering practices like Test Driven Development. And their dedicated QA teams couldn't prevent these quality issues because they were full of junior testers who didn't know what they were doing, didn't automate tests and were ordered to test and pass everything quickly to avoid falling behind schedule. So, poor quality development and QA practices were built into the system development process, and independent QA teams didn't fix it. Independent dedicated QA teams are an outdated and costly approach to quality. It's like a car factory that consistently produces defect-ridden vehicles only to disassemble and fix them later. Instead of testing and fixing features at the end, we should build quality into the process from the start. Modern engineering teams do this by working in cross-functional teams. Teams that use test-driven development approaches to define testable requirements and continuously review, test, and integrate their work. This allows them to catch and address issues early, resulting in faster, more efficient, and higher-quality development. In modern engineering teams, QA specialists are quality champions. Their expertise strengthens the team’s ability to build robust systems, ensuring quality is integral to how the product is built from the outset. The old model, where testing is done after development, belongs in the past. Today, quality is everyone’s responsibility—not through role dilution but through shared accountability, collaboration, and modern engineering practices.

GenAI is delivering productivity gains of up to 20% to the software development lifecycle, and Deloitte’s latest research dives into how GenAI is driving this transformation. Faruk Muratovic, Diana Kearns-Manolatos (she/her), and Ahmed Alibage, CMS®, Ph.D. recently published an insightful report in the IEEE Computer Society’s journal [https://deloi.tt/3TtkCC6]. Their findings highlight not only productivity gains, but also the importance of trust and transparency.     Building trust in GenAI starts with thoughtful human oversight. The report recommends keeping humans-in-the-loop (HITL) to ensure code quality, manage risk, and provide transparency. These key actions stand out:    •Promote design transparency and explainability: By fostering open, iterative design, teams can balance innovation with consistent, high-quality results.  •Strengthen code accuracy with clear metrics: Leveraging repeatable measures like defect density and time-to-delivery helps maintain quality and build confidence in GenAI-driven solutions.  •Create a culture of continuous learning and improvement. As GenAI evolves, teams will stay resilient and innovative.    By taking these actions, tech leaders can help build a future where technology and human expertise go hand in hand—delivering real value, safely and responsibly.   

Step-by-Step Guide to Measuring & Enhancing GCC Productivity - Define it, measure it, improve it, and scale it. Most companies set up Global Capability Centers (GCCs) for efficiency, speed, and innovation—but few have a clear playbook to measure and improve productivity. Here’s a 7-step framework to get you started: 1. Define Productivity for Your GCC Productivity means different things across industries. Is it faster delivery, cost reduction, innovation, or business impact? Pro tip: Avoid vanity metrics. Focus on outcomes aligned with enterprise goals. Example: A retail GCC might define productivity as “software features that boost e-commerce conversion by 10%.” 2. Select the Right Metrics Use frameworks like DORA and SPACE. A mix of speed, quality, and satisfaction metrics works best. Core metrics to consider: • Deployment Frequency • Lead Time for Change • Change Failure Rate • Time to Restore Service • Developer Satisfaction • Business Impact Metrics Tip: Tools like GitHub, Jira, and OpsLevel can automate data collection. 3. Establish a Baseline Track metrics over 2–3 months. Don’t rush to judge performance—account for ramp-up time. Benchmark against industry standards (e.g., DORA elite performers deploy daily with <1% failure). 4. Identify & Fix Roadblocks Use data + developer feedback. Common issues include slow CI/CD, knowledge silos, and low morale. Fixes: • Automate pipelines • Create shared documentation • Protect developer “focus time” 5. Leverage Technology & AI Tools like GitHub Copilot, generative AI for testing, and cloud platforms can cut dev time and boost quality. Example: Using AI in code reviews can reduce cycles by 20%. 6. Foster a Culture of Continuous Improvement This isn’t a one-time initiative. Review metrics monthly. Celebrate wins. Encourage experimentation. Involve devs in decision-making. Align incentives with outcomes. 7. Scale Across All Locations Standardize what works. Share best practices. Adapt for local strengths. Example: Replicate a high-performing CI/CD pipeline across locations for consistent deployment frequency. Bottom line: Productivity is not just about output. It’s about value. Zinnov Dipanwita Ghosh Namita Adavi ieswariya k Karthik Padmanabhan Amita Goyal Amaresh N. Sagar Kulkarni Hani Mukhey Komal Shah Rohit Nair Mohammed Faraz Khan

We know LLMs can substantially improve developer productivity. But the outcomes are not consistent. An extensive research review uncovers specific lessons on how best to use LLMs to amplify developer outcomes. 💡 Leverage LLMs for Improved Productivity. LLMs enable programmers to accomplish tasks faster, with studies reporting up to a 30% reduction in task completion times for routine coding activities. In one study, users completed 20% more tasks using LLM assistance compared to manual coding alone. However, these gains vary based on task complexity and user expertise; for complex tasks, time spent understanding LLM responses can offset productivity improvements. Tailored training can help users maximize these advantages. 🧠 Encourage Prompt Experimentation for Better Outputs. LLMs respond variably to phrasing and context, with studies showing that elaborated prompts led to 50% higher response accuracy compared to single-shot queries. For instance, users who refined prompts by breaking tasks into subtasks achieved superior outputs in 68% of cases. Organizations can build libraries of optimized prompts to standardize and enhance LLM usage across teams. 🔍 Balance LLM Use with Manual Effort. A hybrid approach—blending LLM responses with manual coding—was shown to improve solution quality in 75% of observed cases. For example, users often relied on LLMs to handle repetitive debugging tasks while manually reviewing complex algorithmic code. This strategy not only reduces cognitive load but also helps maintain the accuracy and reliability of final outputs. 📊 Tailor Metrics to Evaluate Human-AI Synergy. Metrics such as task completion rates, error counts, and code review times reveal the tangible impacts of LLMs. Studies found that LLM-assisted teams completed 25% more projects with 40% fewer errors compared to traditional methods. Pre- and post-test evaluations of users' learning showed a 30% improvement in conceptual understanding when LLMs were used effectively, highlighting the need for consistent performance benchmarking. 🚧 Mitigate Risks in LLM Use for Security. LLMs can inadvertently generate insecure code, with 20% of outputs in one study containing vulnerabilities like unchecked user inputs. However, when paired with automated code review tools, error rates dropped by 35%. To reduce risks, developers should combine LLMs with rigorous testing protocols and ensure their prompts explicitly address security considerations. 💡 Rethink Learning with LLMs. While LLMs improved learning outcomes in tasks requiring code comprehension by 32%, they sometimes hindered manual coding skill development, as seen in studies where post-LLM groups performed worse in syntax-based assessments. Educators can mitigate this by integrating LLMs into assignments that focus on problem-solving while requiring manual coding for foundational skills, ensuring balanced learning trajectories. Link to paper in comments.

Over the years, I've collected a lot of material and thoughts about developer productivity: research papers, practitioner articles, cautionary tales, and lessons from my own experience. The topic keeps coming up because our industry keeps getting it wrong. I finally put it all together into one article. The article covers: • Why every simple metric (LOC, velocity, hours worked) fails for the same reasons, explained through the McNamara Fallacy, Goodhart's Law, and a lesser-known phenomenon called surrogation: people nonconsciously replace the goal with the metric, forgetting what the number was supposed to represent in the first place • Why modern frameworks like DORA and SPACE are genuine progress but are routinely misapplied: used for benchmarking and comparison when they were designed for self-improvement • Why the McKinsey developer productivity framework drew the strongest collective rebuttal from the engineering community in recent memory • Why software development is knowledge work, not manufacturing — and why that distinction matters more than any metric • Why AI studies on developer productivity contradict each other wildly, and none of them should be trusted yet • What (IMO) actually works: measuring developer experience (feedback loops, cognitive load, flow state), focusing on team outcomes rather than individual metrics, and thinking in systems rather than individuals The core argument: the real question is not "How productive are our developers?" It is "How do we create the conditions where developers can do their best work?" One requires a dashboard. The other requires leadership. Link to the full article in the first comment. #DeveloperProductivity #SoftwareDevelopment #SoftwareEngineering

📊 Goodhart’s law is why KPIs fail We all know the feeling: you buy a gym membership, the treadmill lights up, the apps track your steps… but somehow, the six-pack never arrives. That’s KPIs in a nutshell. On paper, everything looks active. In reality, not much is effective. Every CEO loves KPIs. They look great in board decks, they make dashboards light up like a Christmas tree, and they give leaders the comforting illusion of control. But here’s the problem: “When a measure becomes a target, it ceases to be a good measure.” That’s Goodhart’s Law, coined in 1975, and it’s still killing strategy today. 🎯 The KPI trap KPIs start out as useful indicators. But the moment they become targets, people game them. • Call centers optimize for call length → agents hang up quickly, not helpfully. • Sales teams optimize for pipeline size → suddenly you’ve got a pipeline full of fantasy deals. • Marketing optimizes for clicks → congratulations, you now have traffic from bots in Bulgaria. Leaders celebrate the numbers, but the outcomes? Not so much. 📊 Research reality • McKinsey found that only 23% of executives believe their KPIs are aligned with strategy. The other 77% are tracking noise, not progress. • Gallup’s global workplace survey shows that only 20% of employees feel their performance metrics are managed in a way that motivates them to do outstanding work. In other words, most KPIs disengage more than they inspire. • Harvard Business Review highlights that vanity metrics reduce performance because they distract leaders from execution. • A London School of Economics study found that when incentives and targets are misaligned, employees “optimize the metric” even if it damages overall outcomes. Classic Goodhart’s Law in action. • Deloitte research shows that companies that focus on “execution-linked metrics” (customer impact, speed, resilience) outperform those that over-index on internal KPIs by up to 60% in shareholder returns. The data is clear: measuring the wrong things is often worse than not measuring at all. Dashboards are like treadmills. They light up, make you feel productive, and give the illusion of progress. But unless leaders ensure the numbers actually translate to outcomes, you’re just running in place. 🛡️ Leadership’s responsibility The KPI problem isn’t technical, it’s leadership. • Leaders confuse measurement with management. • Leaders celebrate dashboards instead of outcomes. • Leaders treat KPIs like strategy, when they should be diagnostics. The CEO’s job isn’t to chase numbers, it’s to ensure numbers reflect reality. 📌 A leadership reflection Goodhart’s Law is alive in every company. Metrics aren’t bad; bad leadership of metrics is. Leaders don’t win by measuring more. They win by measuring what matters, then managing execution. Because at the end of the day, vanity KPIs make dashboards look pretty. But they don’t make companies perform. #Leadership #Strategy #Execution #DecisionMaking #Management