Skip to content

Lab 20 — Quality Planning and the Cost of Quality

At a glance

Learning objectives

By the end of this lab you should be able to:

  1. Convert a vague quality goal into measurable attributes with named mechanisms.
  2. Separate quality assurance from quality control, and assign each to a mechanism.
  3. Decide how much prevention and appraisal work is worth paying for, using arithmetic rather than preference.
  4. Design quality metrics that resist the gaming they will inevitably attract.

Scenario

This lab returns to the Mobile Banking Application from Labs 04 and 16. The release covers balances, transaction history, transfers between the customer's own accounts, bill payment, and card blocking.

Delivery facts:

Item Value
Cost per person-day PKR 20,000
Cost to fix a defect found in review or test Included in the person-day figure
Cost of one defect that reaches production PKR 400,000 (hotfix, support, data repair, remediation, and reputational handling)
Team 3 developers, 1 tester, 1 security specialist at 30%
Regulatory position Authentication requirements must be met before release; the security specialist must sign off
Customer context Approximately 40,000 retail customers; the bank's regulator publishes incident notices

Three quality approaches have been costed by the team. Each changes the number of defects that reach production. All figures are invented for teaching.

Approach Prevention (person-days) Review (person-days) Testing (person-days) Defects reaching production
A — Test-heavy 0 10 40 28
B — Balanced 20 25 30 9
C — Minimal 5 5 20 45

Where "prevention" means requirements review, design review, coding standards, and automated static checks; "review" means structured peer review of every change; and "testing" means unit, integration, system, and acceptance testing.

Your role

You are the project manager. The programme board wants a quality plan, and one member of the board has said that the balanced approach "looks like gold-plating, and the test-heavy approach is cheaper".

Project information and assumptions

The scenario is part of the example-project library — read the full description there if you want the wider context.

  • Labour cost is PKR 20,000 per person-day for all approaches. Do not apply a productivity factor; the board's figures are given in person-days of project work already.
  • Production defect cost is PKR 400,000 each, and is a cash cost to the bank, not an estimate of goodwill.
  • Assume the defect counts given are the only difference between the approaches.
  • Assume the regulatory sign-off is required in all three approaches and is not a differentiator.

Tasks

Task 1 — Define the quality attributes

"High quality" is not usable. Define five quality attributes for this application, in a form that could be measured. For each, complete a row:

Attribute Why it matters here How it would be observed Falsifying observation

At least one attribute must concern data correctness and one must concern availability at peak load. At least one must be about a customer experience the bank has previously received complaints about.

Task 2 — Separate assurance from control

For each attribute in Task 1, name:

  1. One quality assurance mechanism — a change to how work is done.
  2. One quality control mechanism — a check on the product.

Then answer: why does a plan containing only control mechanisms produce more expensive quality over time, even when each individual check is effective?

Task 3 — Cost the three approaches

For each approach, calculate:

  1. Labour cost = (prevention + review + testing) × PKR 20,000.
  2. Failure cost = defects reaching production × PKR 400,000.
  3. Total cost of quality = labour + failure.

Present as a table with the arithmetic visible, then answer:

  1. Which approach has the lowest total cost, and by how much?
  2. Which approach has the lowest labour cost, and which has the lowest total cost — and why are those different answers?
  3. Express each approach's failure cost as a multiple of its labour cost.

Task 4 — Answer the board member

The board member said the balanced approach looks like gold-plating and the test-heavy approach is cheaper. Using your numbers, write a reply of no more than 180 words. It must:

  1. Acknowledge what is correct in the remark (the test-heavy approach genuinely costs less labour).
  2. State the total-cost comparison with figures.
  3. Name what the difference buys, in customer terms rather than in defect terms.
  4. Avoid claiming that prevention always pays — state the condition under which the board member would be right.

Task 5 — Test the sensitivity

The production defect cost is the figure most likely to be wrong. Recalculate the total cost of approaches A and B assuming a production defect costs:

  1. PKR 100,000
  2. PKR 1,500,000

Present all three scenarios (100,000 / 400,000 / 1,500,000) in one table, and state the break-even production defect cost at which A and B become equal. Show your working — you can find it by inspection from your three scenarios, and state that method.

Then answer: what does the break-even point tell you about how much effort to spend arguing about the defect cost estimate versus implementing the quality plan?

Task 6 — Write the definition of done

Write a definition of done for this team with at least ten checkable conditions, covering code review, automated checks, test levels, security sign-off, accessibility, documentation, and deployment readiness.

Then answer:

  1. Which two conditions are the most important for the payment and authentication paths, and why?
  2. Which condition would be waived first under launch pressure, and what would you put in place so that waiving it requires a decision rather than a shrug?

Task 7 — Design the metrics plan

Choose four quality metrics, and for each record:

Metric What decision it supports Who sees it How it could be gamed Counter-measure

At least one metric must be a leading indicator and at least one must measure defect escape, not defect volume.

Then write two sentences on why per-developer defect counts should not be published.

Deliverable

  1. Five quality attributes with observation and falsifying observation.
  2. The QA and QC mechanisms per attribute, and the answer about control-only plans.
  3. The cost-of-quality table with the three answers.
  4. The 180-word reply to the board member.
  5. The sensitivity table, the break-even figure, and the conclusion about arguing over estimates.
  6. A definition of done with at least ten checkable conditions and the two answers.
  7. A metrics plan with four metrics, including one leading indicator and one escape measure.

Hints and guidance

  • An attribute without a falsifying observation is a wish. If nothing you could see would show it unmet, it is not a criterion.
  • Prevention changes how work is done; control checks the product. Both are needed, and they are not substitutes.
  • The cheapest labour is rarely the cheapest quality. The comparison you are being asked to make is total cost, and the failure term dominates unless defects are cheap.
  • Break-even analysis converts an argument into a fact. If the decision does not change across the plausible range of the disputed input, stop arguing about the input.
  • A gamed metric is worse than no metric, because it is believed. Design the counter-measure at the same time as the metric.

Self-check criteria

Check How to verify it
Attributes are measurable Each has an observation and a falsifying observation
QA and QC are distinct No mechanism is listed as both
Labour costs correct A 1,000,000; B 1,500,000; C 600,000
Failure costs correct A 11,200,000; B 3,600,000; C 18,000,000
Totals correct A 12,200,000; B 5,100,000; C 18,600,000
Lowest total identified B, cheaper than A by 7,100,000
Reply acknowledges the correct part It concedes the labour-cost point before arguing the total
Break-even stated with method A figure within the plausible range, with the method named
Definition of done is checkable Each of the ten conditions can be judged met or not met
Metrics include leading and escape measures Both present, with counter-measures

Reflection

  1. Approach B is best on total cost, but has the highest labour cost. What does that tension mean for how you present the plan to a board that sees labour as "the cost"?
  2. Your break-even analysis probably shows the decision is robust across a wide range of defect costs. What does that let you stop doing in meetings?
  3. One of your metrics is an escape measure. What would have to happen for escape rate to fall without quality improving?
  4. The security specialist is at 30%. Which of your quality mechanisms depends on them, and what is your fallback if they are unavailable at the wrong moment?
Check your numbers

Labour cost (person-days × PKR 20,000)

Approach Person-days Labour cost
A — Test-heavy 0 + 10 + 40 = 50 1,000,000
B — Balanced 20 + 25 + 30 = 75 1,500,000
C — Minimal 5 + 5 + 20 = 30 600,000

Total cost of quality

Approach Labour Failure cost Total
A 1,000,000 28 × 400,000 = 11,200,000 12,200,000
B 1,500,000 9 × 400,000 = 3,600,000 5,100,000
C 600,000 45 × 400,000 = 18,000,000 18,600,000

B is cheapest by 7,100,000 against A, despite costing 500,000 more in labour.

Sensitivity

Defect cost A total B total Cheaper
100,000 1,000,000 + 2,800,000 = 3,800,000 1,500,000 + 900,000 = 2,400,000 B
400,000 12,200,000 5,100,000 B
1,500,000 1,000,000 + 42,000,000 = 43,000,000 1,500,000 + 13,500,000 = 15,000,000 B

B is cheaper across the entire plausible range, and the figures diverge rather than converge as defect cost rises. Break-even lies well below PKR 100,000 — you can solve for it exactly: 1,000,000 + 28c = 1,500,000 + 9c → 19c = 500,000 → c ≈ PKR 26,300. Only if a production defect cost less than about 26,000 — roughly a day of labour — would approach A be preferable.

Full working is in the Advanced labs answer key.

Extension challenge

Design a quality programme for a second team working on the same product concurrently, where that team's defects are found by your team's testing. Using the same cost figures, calculate what your team's inspection capacity should be if it were to absorb the second team's escape rate, and what you would need to change in the second team's definition of done to avoid that. Then state the organisational argument you would make, in three sentences, to a manager who says the shared team "already covers testing".