Measuring predictability

The primary measurements for any software development organization are productivity, quality and predictability to measure delivery effectiveness. While the first two are well defined and handled – predictability is still gaining momentum throughout the industry. We are looking at predictability not only in relation to the project cost estimates (Although, properly capturing and handling historic data and reviewing estimates during retrospective can result in some quick wins. Defining ranges for the estimates and their confidence levels along with calibrating them against past data will make cost estimates predictability higher) but predictability extended to the full project delivery life cycle. Of course, in this sense predictability is not completely independent from productivity and quality reflecting their consistency as well. Please, keep in mind that we are talking about project level measurements, not individual Sprints.

At TradePMR, in a manner similar to Ness Predictability Index, we came up with the components of our predictability measures (with different weights, these may have to be adjusted for other environments)

·        Scope variance

·        Schedule variance

·        Business value variance

·        Project burn down variance

·        Release velocity variance

·        Lead time variance

·        Deployment frequency variance

·        Quality gate data variance

·        Defect slippage rate variance

Most of these are self-explanatory and fortunately the underlying data is usually being collected. A few clarifications about less obvious ones: Business value is a relative number used at the project and feature level for proper prioritization (using WSJF or similar framework); Quality gate data for us are the results of code quality scans together with static and dynamic code security scans; Defect slippage – is the number of defects being caught in production and not test.

Capturing and tracking different dimensions of the predictability index together with overall trends came up as a handy mechanism to improve overall consistency and cadence within our project portfolio.

Trade-PMR, Inc., member FINRA/SIPC