Lesson 1 — Software Project Metrics¶
At a glance
- Module: 11 — Metrics, Monitoring & Control
- Level: Advanced
- Estimated time: 45 minutes
- Example project: the analytics dashboard
Learning objectives¶
By the end of this lesson you should be able to:
- Distinguish a measure from a metric, and leading from lagging indicators.
- Derive metrics from goals using the Goal–Question–Metric approach.
- Calculate schedule and cost performance using earned value.
- Explain how metrics are corrupted, and design measures that resist it.
Prerequisites¶
Introduction¶
A project produces numbers constantly: tasks closed, hours spent, defects found, releases shipped. Almost none of them tell you whether the project is in trouble. That is the job of a metric.
The distinction is not pedantic. A measure is a raw observation; a metric is a measure with a purpose attached — it answers a question someone acts on. Projects drown in measures while starving for metrics.
1. Measures, metrics, and indicators¶
| Definition | Example | |
|---|---|---|
| Measure | An observation | 34 story points completed |
| Metric | A measure connected to a question and a decision | Velocity, compared to the forecast |
| Leading indicator | Predicts a future outcome | Number of unresolved dependencies |
| Lagging indicator | Confirms an outcome after the fact | Defects found in production |
Lagging indicators are easy to collect and useless for control: by the time they move, the outcome has happened. Leading indicators are harder and more valuable. Most mature metrics programmes pair one of each — a leading indicator to steer by, a lagging one to confirm the guess.
2. Deriving metrics from goals¶
Goal–Question–Metric (GQM) starts from the decision, not the data.
- Goal — "Reduce the time from requirement to production for small changes."
- Question — "How long does a change take now, and where does it wait?"
- Metric — Lead time per change; proportion of lead time spent waiting for review.
Collecting the metric before deciding the goal is how projects end up with dashboards nobody reads. The test is: what decision would change if this number moved? If there is no answer, do not collect it.
3. Earned value¶
Earned value management (EVM) compares three quantities at a point in time.
| Quantity | Meaning |
|---|---|
| Planned value (PV) | Budgeted cost of the work scheduled by now |
| Earned value (EV) | Budgeted cost of the work actually completed |
| Actual cost (AC) | What has actually been spent |
From these come four derived indicators:
| Indicator | Formula | Reads as |
|---|---|---|
| Schedule variance (SV) | EV − PV | Value of work ahead or behind plan |
| Cost variance (CV) | EV − AC | Value of work gained or lost per spend |
| Schedule performance index (SPI) | EV ÷ PV | Progress rate; 1.0 is on plan |
| Cost performance index (CPI) | EV ÷ AC | Cost efficiency; 1.0 is on plan |
Forecasts follow: EAC = BAC ÷ CPI estimates final cost using current efficiency, where BAC is the budget at completion. VAC = BAC − EAC is the projected overrun.
Two cautions. EVM needs a meaningful way to measure completion, which for software is the hard part. And EVM measures conformance to plan, not value delivery: a project can score perfectly while building something nobody wants. Use it for control, not for justification.
4. How metrics get corrupted¶
When a measure becomes a target, it ceases to be a good measure — a phenomenon named after Goodhart's observation and restated by Strathern. In practice:
| Metric used as target | Predictable response |
|---|---|
| Lines of code written | Verbose code; code that should be deleted is kept |
| Defects found | Fewer defects reported |
| Velocity | Point inflation |
| Percent complete | Early optimism, then a plateau, then a cliff |
Good metrics are collected where the data already exists, are used at team and process level, and are read as trends rather than absolute values. And when a metric is used to judge individuals, you should assume the data is wrong — not because people are dishonest, but because they are human, and the incentive to report favourably is structural.
Key terms¶
| Term | Meaning |
|---|---|
| Measure | A raw observation |
| Metric | A measure connected to a question and a decision |
| Leading indicator | Predicts an outcome before it occurs |
| Lagging indicator | Confirms an outcome after it occurs |
| GQM | Goal–Question–Metric, deriving metrics from decisions |
| PV / EV / AC | Planned value, earned value, actual cost |
| SPI / CPI | Schedule and cost performance indices |
| EAC / VAC | Estimate at completion; variance at completion |
| Goodhart's law | When a measure becomes a target, it stops being a good measure |
Worked example¶
Task: assess the illustrative analytics dashboard project at the end of week 8. All figures are invented for teaching.
Step 1 — Collect the three base quantities. Budget at completion: PKR 100,000 (in thousands — read as any unit you like). Planned to have completed 40% of the work by week 8, so PV = 40,000. Actually completed 30%, so EV = 30,000. Spent AC = 35,000.
Step 2 — Compute variance.
| Indicator | Calculation | Value | Reading |
|---|---|---|---|
| SV | 30,000 − 40,000 | −10,000 | Behind plan by 10,000 of value |
| CV | 30,000 − 35,000 | −5,000 | Over budget by 5,000 of value |
| SPI | 30,000 ÷ 40,000 | 0.75 | Working at three-quarters of the planned rate |
| CPI | 30,000 ÷ 35,000 | 0.86 | Getting 86 paise of value per rupee spent |
Step 3 — Forecast. EAC = 100,000 ÷ 0.86 ≈ 116,700. VAC = 100,000 − 116,700 = −16,700 — a projected overrun of roughly 17% if the current rate continues.
Step 4 — Interpret before reacting. EVM describes what happened, not why. A plausible cause here is the delay found in Module 10's worked example — the interface slip, which is an external dependency, plus the analyst overload absorbing review time. The response depends on the cause: an external delay calls for a schedule response; sustained cost inefficiency calls for a scoping one.
Step 5 — Add the question EVM cannot answer. Is the completed 30% the valuable 30%? If the completed work is the reporting engine and the unstarted work is the visualisation that users actually wanted, the project is worse than the numbers suggest. This is why EVM is paired with a product-level question — has anything shipped that a user can use? — which is why Module 04's incremental delivery matters beyond agile dogma.
Step 6 — Report the trend, not the snapshot. SPI fell from 0.95 to 0.88 to 0.75 across three reporting periods. The trend is visible in week 6; the number alone is only clear in week 8. Report the trend early, because that is when a decision is still cheap.
Common misconceptions¶
“Percent complete is a progress measure”
It is an opinion, and it is systematically optimistic early and unstable late — tasks tend to report 90% for long stretches. Prefer counting completed, verifiable items and using earned value.
“SPI below 1 means the project will be late”
It means work is behind plan. That may resolve, and "behind plan" may simply mean the plan was wrong. SPI also cannot exceed 1 in the final period, which makes extrapolation fragile.
“More metrics mean more control”
Beyond a handful, attention is divided and metrics compete. Choose the few that map to real decisions and retire the rest.
Practical activity¶
Build a metric set. For one example project:
- Write two goals, and derive one question and one metric from each.
- For each metric, state the decision that would change if it moved.
- Add one leading and one lagging indicator.
- Using invented figures, compute SV, CV, SPI, CPI, and a forecast EAC.
- Identify how one of your metrics could be gamed, and adjust it to make that harder.
What "good" looks like: every metric traceable to a decision, at least one leading indicator, a correct EVM computation, and a gaming risk you identified against your own design.
Self-assessment¶
- What is the difference between a measure and a metric, and why does it matter?
- EV = 60, PV = 80, AC = 50. Compute SPI and CPI and interpret both in one sentence.
- Why are leading indicators more useful for control than lagging indicators?
-
Why does using a metric to rank individuals usually destroy the metric?
-
Apply it — earned value, illustrative figures. A six-month project has a BAC of 1,000,000. At the end of month 4 the figures are PV 400,000, EV 340,000, AC 380,000. The last status report also says the team is “85% complete”. (These figures are invented for practice; the arithmetic is what matters.)
Hint: A strong answer computes all four indices, forecasts from them, and separates conformance to plan from delivered value.
Show answers
- A measure is an observation; a metric is a measure attached to a question and a decision. The distinction matters because collecting measures without decisions produces dashboards that consume effort and change nothing.
- SPI = 60 ÷ 80 = 0.75 — behind plan; CPI = 60 ÷ 50 = 1.20 — spending efficiently. The project is getting good value per unit spent but has completed less than planned.
- Because they move before the outcome is fixed, so there is still time to act. Lagging indicators confirm what has already happened, which is useful for learning but not for control.
- Because it makes reporting unfavourable data personally costly, so unfavourable data stops being reported. The metric then measures willingness to report rather than the thing itself.
- SV = EV − PV = 340,000 − 400,000 = −60,000 (behind schedule). CV = EV − AC = 340,000 − 380,000 = −40,000 (over budget). SPI = EV ÷ PV = 340,000 ÷ 400,000 = 0.85. CPI = EV ÷ AC = 340,000 ÷ 380,000 ≈ 0.89. Forecast: EAC ≈ BAC ÷ CPI = 1,000,000 ÷ 0.89 ≈ 1.12 million, so about 120,000 more than planned; and at the current rate the duration forecast is 6 ÷ 0.85 ≈ 7.1 months, about a month late. Tell the sponsor that we are earning 85% of the planned value at about 89% efficiency, and that both forecasts assume the current rate continues against a stable baseline. Then add the caveat that matters: EVM measures conformance to plan, not delivered value, and “85% complete” is not evidence of anything unless it is defined as a verifiable measure — pair it with a product measure such as accepted increments or the defect trend.
Summary¶
- Measures become metrics only when tied to a decision; GQM derives them from goals.
- Pair leading indicators for control with lagging indicators for confirmation.
- Earned value compares PV, EV, and AC to give SV, CV, SPI, and CPI, and to forecast final cost.
- EVM measures conformance to plan, not delivered value; pair it with a product-level question.
- Metrics used as targets get corrupted; collect from existing data, use trends, and never rank individuals.
Further learning¶
- Lesson 2 — Monitoring and Control
- A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Project Management Institute — measurement and monitoring.
- Measuring and Managing Performance in Organizations, Robert D. Austin — on measurement dysfunction.
- Accelerate: The Science of Lean Software and DevOps, Nicole Forsgren, Jez Humble, and Gene Kim — on software delivery metrics and their research basis.
Next¶
Continue to Lesson 2 — Monitoring and Control.