Do You Know What’s Driving Your Program Performance Metrics?
Want to know what is driving your program performance metrics? Program and project metrics are fundamental to understanding if your program/project is achieving its objectives. Let’s see what we might learn.
Starting Point
I have seen a number of program metrics being shared, via status reports, without the audience really understanding, or asking probing questions about, the information being shared. Let’s use the current U.S. unemployment issue as a sample program. If you were the program executive responsible for fixing the U.S. unemployment issue, you would probably want to know the answer to a few questions:
1. What is the unemployment rate today? What is the current trend?
2. What are the drivers for calculating the unemployment rate? What are the trends of those drivers?
3. As you introduce changes, what impact do they have on the metrics of interest?
http://www.hendon-group.com/services-information/program-metrics-and-analytics-offering/
The answer to the first question is illustrated in Figure 1 “U.S. Unemployment Rate.”
- U.S. Unemployment rate – unadjusted – chart
Figure 1 “U.S. Unemployment Rate (Unadjusted).
As we can see, the U.S. Unemployment rate (unadjusted) remains on an upward trend. This is bad. Since the November 2008 election, the unemployment rate has been on an accelerated upward path and is hovering in the 9+ percentage range. To really call the unemployment issue fixed, one would expect to see a rate somewhere around 4 – 5 percent.
Program Performance Metrics: Calculations and Trends
The second set of questions you would want to know the answer to are related to the calculation of the unemployment rate and related trends. The unemployment rate is calculated by dividing the unemployment level by the civilian labor force level. Let’s look at those two numbers and their trends. The unemployment levels are illustrated in Figure 2 “Unemployment Levels (Unadjusted).” The civilian labor force levels are illustrated in Figure 3 “Civilian Labor Force Levels (Unadjusted).”
- U.S. unemployment level – 2001 – 2011 – unadjusted
Figure 2 “U.S. Unemployment Level (Unadjusted)
- U.S. Civilian labor force levels – 2001 – 2011 (unadjusted)
Figure 3 “U.S. Civilian Labor Force level (Unadjusted).
So let’s test a calculation and confirm we can calculate the reported unemployment rate. The unemployment rate equals “Unemployment Level (Unadj)”/”Civilian Labor Force Level (Unadj).” Calculating this number for January 2011 yields the following: 14.9 Million/152.5 Million or 9.8%. This matches the January 2011 reported number of 9.8%.
The calculation also tells us two things about the unemployment rate. If the civilian labor force remains constant, an increase in the unemployment level will increase the unemployment rate. The second thing this calculation tells us is if the unemployment levels remain unchanged and the civilian labor force level decreases (as a result of people giving up on finding work), the employment rate will decrease. For example, if people are no longer looking for work, these individuals will not be reported in the unemployment levels and, at the same time, reduce the civilian workforce number. This phenomenon would show a false sense that the unemployment rate is declining when in fact the unemployment rate may have a slight increase or remain unchanged.
Program Performance Metrics: Closing Remarks
The program manager must understand what data is needed to communicate accurately the current and future state of a program. The consumer of the information must make sure the information being shared answers their questions about the program/project and not necessarily accept the data at face value. The consumer must ask probing questions and assess the information for reasonableness.
Know what program data you need to understand how the program is really doing. The next time you receive program data on one of your programs, do not simply glance at the information. Take some time to understand what the data is telling you. If you are only looking at point-in-time data, ask the program manager to put the data in chart form so you can examine trends. If key data is calculated, also look at the data used in the calculations and examine their trends. You may be surprised what you learn about your program.
If you are a sports fan, you have probably heard of Wayne Gretzky and Michael Jordon. Both of these superstar athletes had one thing in common. They would move to where the puck (or basketball) was going to be. Program/project leadership teams need to do the same thing. They need to know where the program/project is going to be and recommend and/or take action early enough to fix the issues so they do not stall the program and face cost or schedule over-runs.
I have seen instances where a program manager either blindly reports program data because that is what they think the leadership team wants to review, but the data does not tell the real story about the program. I have also seen program managers report a program is on track because they are looking at the wrong data. When, in fact, just the opposite is true (the program is about to go over the cliff within the next 60 – 90 days.) Without having a good understanding of metrics and their trends, a false sense of confidence (short-lived) could be realized.
So, determine what metrics are required to tell the true story about a program. Establish these program metrics early and integrate them into the program status reporting process. As preparation for reporting on these metrics, understand the drivers of the metrics and their trends. Provide driver and trend information at least as backup material for your status reporting. You will gain great insight from the program/project metrics, which will help you manage risks and issues earlier. You will be glad you did.
Author Ira M. Hendon, MBA, PMP® is an expert in program/project leadership consulting and protecting intellectual property. For a no-risk consultation check out Hendon Group at the following link for more information and expert advice on program performance metrics.