Developer Productivity Metrics

You're a #CTO. Your board asks: "What's our ROI on AI coding tools?" Your answer: "40% of our code is AI-generated!" They respond: "So what? Are we shipping faster? Are customers happier?" Most CTOs are measuring AI impact completely wrong. Here's what some are tracking: - Percentage of AI-generated code - Developer hours saved per week - Lines of code produced - AI tool adoption rates These metrics are like measuring how fast your assembly line workers attach parts while ignoring whether your cars actually start. Here's what you SHOULD measure instead: 1. Delivered business value 2. Customer cycle time 3. Development throughput 4. Quality and reliability 5. Total cost of delivery (not just development) 6. Team satisfaction Software development isn't a typing competition—it's a complex system. If AI makes your developers 30% faster but your deployment takes 2 weeks and QA adds another week, your customer delivery improves by maybe 7%. You've speed up the wrong part. The solution: A/B test your teams. Give half your teams AI tools, measure business outcomes over 2-3 release cycles. Track what customers actually experience, not how much developers produce. Companies that measure business impact from AI will pull ahead. Those measuring vanity metrics will wonder why their expensive tools aren't moving the needle. Stop measuring how much code AI generates. Start measuring how much faster you deliver value to customers. What are you actually measuring? And is it moving your business forward? -> Follow me for more about building great tech organizations at scale. More insights in my book "All Hands on Tech"

Has Amazon cracked the code on developer productivity with its cost to serve software (CTS-SW) metric? Amazon applied its well-known "working backwards" methodology to developer productivity. "Working backwards" in this case starting with the outcome: concrete returns for the business. This is measured by looking at the rate of customer-facing changes delivered by developers, i.e. "what the team deems valuable enough to review, merge, deploy, and support for customers", in the words of the blog post by Jim Haughwout https://lnkd.in/eqvW5wbi . This metric is different from other measures of developer productivity which look only at velocity or time saved. Instead, "CTS-SW directly links investments in the developer experience to those outcomes by assessing how frequently we deliver new or better experiences. Some organizations fall into the anti-pattern of calculating minutes saved to measure value, but that approach isn’t customer-centered and doesn’t prove value creation." This aligns with Gartner's own research on developer productivity. In our 2024 Software Engineering survey, we asked what productivity metric organizations are using to measure their developers. We also asked about a basket of ten success metrics, including software usability, retention of top performers, and meeting security standards. This allowed us to find out which productivity metric was associated most with success. What we found in our survey was that *rate of customer-facing changes* is the metric most associated with success. Some other productivity metrics were actually *negative associated* with success. But *rate of customer-facing changes* is what organizations should focus on. Sadly, our survey found that few organizations (just 22%) use this metric. I presented this data at our #GartnerApps summit [and the next summit is coming up in September: https://lnkd.in/ey2kpc2 ] Every metrics gets gamed. So I always recommend "gaming the gaming". A developer might game the CTS-SW metric by focusing more on customer-facing changes. But... this is actually a good thing. You're gaming the gaming. We will be watching closely how this metric gets adopted alongside DORA, SPACE, and other metrics in the industry.

Probably the simplest most-effective way to improve productivity is to reduce your work in progress (things you work on simultaneously) to 1. Think about a situation where you must work with a "platform team." Your team is bopping along until it comes across something it needs to do that the platform can't handle. It then stops work and hands off to the platform team. Rather than being idle while it waits, the first team now starts working on a second thing until it needs a database change, which it hands off to the database team. Not wanting to be idle, it starts working on a third thing. Weinberg points out that every "thing" you work on reduces productivity by about 20%. So, if you have three 5-day tasks. Working on two of them at once adds 20% to each task, so it will take 12 days to do 10 days of work. Add a third task and we're adding 2 days to each task, so it now will take 21 days to do 15 days of work. This isn't even considering what happens if the other team gets it wrong and you need to resubmit the request or the fact that it now takes up to four times longer (21 days rather than 5) to get something useful into your customer's hands. So, to work on only one thing at a time, we need to eliminate the dependencies. Our single product team needs to be able to make platform and database changes (safe ones, at least, to avoid collisions with other teams). They need to align with the other teams when they make those changes so that they don't break anything, but I find that an occasional chapter/guild meeting to deal with consistency issues takes way less time than the time you lose to WIP>1.

🚀 𝐓𝐡𝐞 2024 𝐒𝐭𝐚𝐭𝐞 𝐨𝐟 𝐃𝐞𝐯𝐎𝐩𝐬 𝐑𝐞𝐩𝐨𝐫𝐭 𝐢𝐬 𝐎𝐮𝐭! 🚀 The latest 𝐃𝐎𝐑𝐀 (DevOps Research and Assessment) report has dropped, packed with insights on how 𝐀𝐈 and 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 are reshaping 𝐃𝐞𝐯𝐎𝐩𝐬 and 𝐬𝐨𝐟𝐭𝐰𝐚𝐫𝐞 𝐝𝐞𝐥𝐢𝐯𝐞𝐫𝐲 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞. From faster deployment times to the unexpected effects of AI. Here are the 𝐤𝐞𝐲 𝐡𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: 📊 𝐓𝐡𝐞 𝐅𝐨𝐮𝐫 𝐊𝐞𝐲 𝐌𝐞𝐭𝐫𝐢𝐜𝐬: A Blueprint for High Performance DORA's foundation rests on four metrics that define software delivery excellence: 𝐥𝐞𝐚𝐝 𝐭𝐢𝐦𝐞 𝐟𝐨𝐫 𝐜𝐡𝐚𝐧𝐠𝐞, 𝐝𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭 𝐟𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐲, 𝐜𝐡𝐚𝐧𝐠𝐞 𝐟𝐚𝐢𝐥 𝐫𝐚𝐭𝐞, and 𝐫𝐞𝐜𝐨𝐯𝐞𝐫𝐲 𝐭𝐢𝐦𝐞. This year's report shows that teams who excel across these metrics achieve elite performance, 𝐫𝐞𝐠𝐚𝐫𝐝𝐥𝐞𝐬𝐬 𝐨𝐟 𝐢𝐧𝐝𝐮𝐬𝐭𝐫𝐲. 👉 It’s proof that high performance depends not on sector but on 𝐬𝐦𝐚𝐫𝐭, 𝐞𝐟𝐟𝐞𝐜𝐭𝐢𝐯𝐞 𝐃𝐞𝐯𝐎𝐩𝐬 𝐩𝐫𝐚𝐜𝐭𝐢𝐜𝐞𝐬. 🤖 𝐀𝐈 𝐀𝐝𝐨𝐩𝐭𝐢𝐨𝐧 = 𝐇𝐢𝐠𝐡𝐞𝐫 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲... with a Twist 𝐀𝐈 has gone 𝐦𝐚𝐢𝐧𝐬𝐭𝐫𝐞𝐚𝐦, with 81% of organizations now integrating it into 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭 workflows. DevOps professionals report big 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲 𝐛𝐨𝐨𝐬𝐭𝐬, but there’s a paradox: AI speeds up "𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞" work but leaves more time for the 𝐫𝐞𝐩𝐞𝐭𝐢𝐭𝐢𝐯𝐞 𝐭𝐚𝐬𝐤𝐬 we’re all too familiar with. 👉 Meaning AI makes the 𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞 work 𝐪𝐮𝐢𝐜𝐤𝐞𝐫, but it 𝐝𝐨𝐞𝐬𝐧’𝐭 𝐞𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐞 everything we 𝐝𝐨𝐧’𝐭 𝐰𝐚𝐧𝐭 𝐭𝐨 𝐝𝐨 😢! 🛠️ 𝐏𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠: Powers Developer Independence 𝐈𝐧𝐭𝐞𝐫𝐧𝐚𝐥 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐫 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦𝐬 are transforming team workflows by enabling self-service. Teams with strong platforms report a 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲 increase of 8% and 𝐭𝐞𝐚𝐦 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 boost of 10%! However, with more layers in the pipeline, overall delivery speed can sometimes slow. 👉 For the best results, 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 should focus on 𝐮𝐬𝐞𝐫-𝐜𝐞𝐧𝐭𝐞𝐫𝐞𝐝 design, developer 𝐢𝐧𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐜𝐞, and a 𝐩𝐫𝐨𝐝𝐮𝐜𝐭-𝐨𝐫𝐢𝐞𝐧𝐭𝐞𝐝 approach. ❤️🔥 𝐓𝐡𝐞 𝐑𝐨𝐥𝐞 𝐨𝐟 𝐋𝐞𝐚𝐝𝐞𝐫𝐬𝐡𝐢𝐩 & 𝐒𝐭𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐥𝐞𝐚𝐝𝐞𝐫𝐬𝐡𝐢𝐩 that prioritizes stability and 𝐮𝐬𝐞𝐫-𝐜𝐞𝐧𝐭𝐫𝐢𝐜𝐢𝐭𝐲 is a game-changer. Teams with consistent 𝐩𝐫𝐢𝐨𝐫𝐢𝐭𝐢𝐞𝐬 and 𝐮𝐬𝐞𝐫-𝐟𝐨𝐜𝐮𝐬𝐞𝐝 goals produce better software, experience less burnout, and enjoy higher job satisfaction. 👉 Set the 𝐠𝐨𝐚𝐥 for your organization and teams to be just a 𝐥𝐢𝐭𝐭𝐥𝐞 𝐛𝐞𝐭𝐭𝐞𝐫 than yesterday. 𝐓𝐡𝐞 𝐌𝐚𝐢𝐧 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲: 💡 AI and platform engineering are powerful, but they’re not magic solutions. Elite performance is about 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭, 𝐞𝐱𝐩𝐞𝐫𝐢𝐦𝐞𝐧𝐭𝐢𝐧𝐠, and staying 𝐮𝐬𝐞𝐫-𝐟𝐨𝐜𝐮𝐬𝐞𝐝. 📎 Link to the full report is in the comments! #DevOps #AI #PlatformEngineering #StateOfDevOps #DORA #Leadership

As a client project manager, I consistently found that offshore software development teams from major providers like Infosys, Accenture, IBM, and others delivered software that failed 1/3rd of our UAT tests after the provider's independent dedicated QA teams passed it. And when we got a fix back, it failed at the same rate, meaning some features cycled through Dev/QA/UAT ten times before they worked. I got to know some of the onshore technical leaders from these companies well enough for them to tell me confidentially that we were getting such poor quality because the offshore teams were full of junior developers who didn't know what they were doing and didn't use any modern software engineering practices like Test Driven Development. And their dedicated QA teams couldn't prevent these quality issues because they were full of junior testers who didn't know what they were doing, didn't automate tests and were ordered to test and pass everything quickly to avoid falling behind schedule. So, poor quality development and QA practices were built into the system development process, and independent QA teams didn't fix it. Independent dedicated QA teams are an outdated and costly approach to quality. It's like a car factory that consistently produces defect-ridden vehicles only to disassemble and fix them later. Instead of testing and fixing features at the end, we should build quality into the process from the start. Modern engineering teams do this by working in cross-functional teams. Teams that use test-driven development approaches to define testable requirements and continuously review, test, and integrate their work. This allows them to catch and address issues early, resulting in faster, more efficient, and higher-quality development. In modern engineering teams, QA specialists are quality champions. Their expertise strengthens the team’s ability to build robust systems, ensuring quality is integral to how the product is built from the outset. The old model, where testing is done after development, belongs in the past. Today, quality is everyone’s responsibility—not through role dilution but through shared accountability, collaboration, and modern engineering practices.

Stop chasing waterfall (and vanity metrics)! Forget vanity metrics and focus on 4 simple Flow Metrics. Vanity metrics like velocity or the number of commits or pull request reviews by developer, can do more harm than good. "What gets measured, gets managed" Which means, what gets measured gets gamed - and developers are some really smart people who quickly learn to game the system. Flow Metrics are in your system anyway and can help you create a better narrative around metrics. You are not measuring individual contributions. You are not comparing one team with another. You simply want to create a more stable and system - by improving the flow of work. Here are the 4 Flow Metrics: -> Work In Progress: The number of work items started but not finished. Too much WIP? Expect delays, context-switching, and all the madness that follows. ->Throughput: The number of work items finished per unit of time. Think of it as a speedometer for value delivery. -> Work Item Age: The amount of elapsed time between when a work item started and the current time. High values here? Work is probably waiting around longer than it’s getting done. A crucial measure for predictability. -> Cycle Time: The amount of elapsed time between when a work item started and when a work item finished. How long work takes from start to finish - gives you an idea to determine "when it will be done" Follow me for more tips on improving your ways of working!

We know LLMs can substantially improve developer productivity. But the outcomes are not consistent. An extensive research review uncovers specific lessons on how best to use LLMs to amplify developer outcomes. 💡 Leverage LLMs for Improved Productivity. LLMs enable programmers to accomplish tasks faster, with studies reporting up to a 30% reduction in task completion times for routine coding activities. In one study, users completed 20% more tasks using LLM assistance compared to manual coding alone. However, these gains vary based on task complexity and user expertise; for complex tasks, time spent understanding LLM responses can offset productivity improvements. Tailored training can help users maximize these advantages. 🧠 Encourage Prompt Experimentation for Better Outputs. LLMs respond variably to phrasing and context, with studies showing that elaborated prompts led to 50% higher response accuracy compared to single-shot queries. For instance, users who refined prompts by breaking tasks into subtasks achieved superior outputs in 68% of cases. Organizations can build libraries of optimized prompts to standardize and enhance LLM usage across teams. 🔍 Balance LLM Use with Manual Effort. A hybrid approach—blending LLM responses with manual coding—was shown to improve solution quality in 75% of observed cases. For example, users often relied on LLMs to handle repetitive debugging tasks while manually reviewing complex algorithmic code. This strategy not only reduces cognitive load but also helps maintain the accuracy and reliability of final outputs. 📊 Tailor Metrics to Evaluate Human-AI Synergy. Metrics such as task completion rates, error counts, and code review times reveal the tangible impacts of LLMs. Studies found that LLM-assisted teams completed 25% more projects with 40% fewer errors compared to traditional methods. Pre- and post-test evaluations of users' learning showed a 30% improvement in conceptual understanding when LLMs were used effectively, highlighting the need for consistent performance benchmarking. 🚧 Mitigate Risks in LLM Use for Security. LLMs can inadvertently generate insecure code, with 20% of outputs in one study containing vulnerabilities like unchecked user inputs. However, when paired with automated code review tools, error rates dropped by 35%. To reduce risks, developers should combine LLMs with rigorous testing protocols and ensure their prompts explicitly address security considerations. 💡 Rethink Learning with LLMs. While LLMs improved learning outcomes in tasks requiring code comprehension by 32%, they sometimes hindered manual coding skill development, as seen in studies where post-LLM groups performed worse in syntax-based assessments. Educators can mitigate this by integrating LLMs into assignments that focus on problem-solving while requiring manual coding for foundational skills, ensuring balanced learning trajectories. Link to paper in comments.

Taking breaks is part of the job. If you plough straight from task to task, stress builds and focus drops. I'm often guilty of this. I get absorbed by a challenge or an opportunity, dive in and find that three hours have passed before I know it. Microsoft ran EEG tests on people in back-to-back 30-minute meetings. measuring what happens in their brains. They found that short pauses prevented stress from accumulating, boosted engagement, and smoothed the stressful “gear-change” between meetings. In other words, breathers help you do better work. Here are three ways I make breaks count: 1. The pre-task pause Before a tricky task, I go out and take a five-minute walk - even if it's pouring! - then start. Beginning with a breath of fresh air calms the transition and stops me white-knuckling through the first half hour. 2. The one-song reset I turn up the volume on a three-minute track (currently something by Post Malone) stand up, stretch my wrists, look at something out of the window very far away. Then I refill my glass with cold water, and sit back down as the song ends. The music is my timer, so there’s no alarm faff - and I always come back on cue. 3. The park-it technique I end a deep-work stint by writing two lines on the notepad by my keyboard: “what I did” and “what I’ll do next”. Then I step away. Writing down the next step eases my fear of losing momentum, so I can pick it up again the next day. If, like me, you get absorbed and let hours disappear, try one of these this week. What’s your most reliable reset?

Step-by-Step Guide to Measuring & Enhancing GCC Productivity - Define it, measure it, improve it, and scale it. Most companies set up Global Capability Centers (GCCs) for efficiency, speed, and innovation—but few have a clear playbook to measure and improve productivity. Here’s a 7-step framework to get you started: 1. Define Productivity for Your GCC Productivity means different things across industries. Is it faster delivery, cost reduction, innovation, or business impact? Pro tip: Avoid vanity metrics. Focus on outcomes aligned with enterprise goals. Example: A retail GCC might define productivity as “software features that boost e-commerce conversion by 10%.” 2. Select the Right Metrics Use frameworks like DORA and SPACE. A mix of speed, quality, and satisfaction metrics works best. Core metrics to consider: • Deployment Frequency • Lead Time for Change • Change Failure Rate • Time to Restore Service • Developer Satisfaction • Business Impact Metrics Tip: Tools like GitHub, Jira, and OpsLevel can automate data collection. 3. Establish a Baseline Track metrics over 2–3 months. Don’t rush to judge performance—account for ramp-up time. Benchmark against industry standards (e.g., DORA elite performers deploy daily with <1% failure). 4. Identify & Fix Roadblocks Use data + developer feedback. Common issues include slow CI/CD, knowledge silos, and low morale. Fixes: • Automate pipelines • Create shared documentation • Protect developer “focus time” 5. Leverage Technology & AI Tools like GitHub Copilot, generative AI for testing, and cloud platforms can cut dev time and boost quality. Example: Using AI in code reviews can reduce cycles by 20%. 6. Foster a Culture of Continuous Improvement This isn’t a one-time initiative. Review metrics monthly. Celebrate wins. Encourage experimentation. Involve devs in decision-making. Align incentives with outcomes. 7. Scale Across All Locations Standardize what works. Share best practices. Adapt for local strengths. Example: Replicate a high-performing CI/CD pipeline across locations for consistent deployment frequency. Bottom line: Productivity is not just about output. It’s about value. Zinnov Dipanwita Ghosh Namita Adavi ieswariya k Karthik Padmanabhan Amita Goyal Amaresh N. Sagar Kulkarni Hani Mukhey Komal Shah Rohit Nair Mohammed Faraz Khan

How do you measure developer productivity? It's always been a tricky thing for me, but I think this 👇🏼 could be a good solution. According to Emilio Salvador, VP of Developer Relations & Community at GitLab, 3 metrics are key: 1. Task-based: it's not about measuring the NUMBER of tasks a developer completes, but about the TYPE of tasks completed. Some tasks require advanced critical thinking or outside-the-box thinking. They should be evaluated as such. 2. Time-based: measuring the time needed to complete tasks and release features. Using Google's DORA framework to measure both production and deployment times helps identify process weaknesses and bottlenecks. 3. Team-based: no developer works isolated from their colleagues. Measuring the team's delivery performance in terms of business outcomes gives an indication on the productivity of developers. -- Combining these 3 metrics would help engineering managers have a broader view on how the work environment is helping/hindering developer productivity. I would add one more "human" dimension: how the presence of a developer in a team affects the whole team. Factors such as helping teammates, coming up with new concepts, or being proactive on process enhancement count towards a developer's productivity. How do you go about measuring productivity in your company? Share your thoughts!