Lecture 08 — Testing Before Trusting

Learning objectives

• Design test cases from requirements, not from the code
• Choose boundary and edge cases deliberately
• Write simple assert-based checks

Case-based lecture: you attempt, we discuss, then we reveal.

The plan

• Four cases: cs-029, cs-030, cs-031, cs-032
• Each: 5-minute attempt → discussion → reveal → prompt
• Worked examples in between — ask anything
• Break mid-lecture · exit question at the end

Key definitions

• Test case — one input plus the output the specification demands.
• Test table — a systematic list of cases grouped by purpose: normal, boundary, adversarial.
• Normal case — typical use the code was written for.
• Boundary case — an input exactly at a rule's edge (0, empty, the allowed maximum).
• Adversarial case — an input designed to break careless code (negatives, junk text, huge values).
• Black-box testing — designing cases from the specification alone.

The triangle classifier: building the test table

Expected: six verdict lines matching the reference

Cost: O(1) per input; the intellectual work is the table, not the branches.

Black box vs white box on the fee function

def classify(a, b, c):
    sides = sorted([a, b, c])
    if sides[0] + sides[1] <= sides[2]:
        return "invalid"
    if a == b == c:
        return "equilateral"
    if a == b or b == c or a == c:
        return "isosceles"
    return "scalene"

print(classify(3, 4, 5), classify(5, 5, 5), classify(4, 4, 7), classify(1, 2, 8))

Expected: rows ending with a disagreement exactly at x = 3

Cost: Suite size stays small (4-8 rows) because white-box targeting hits the seams instead of sampling blindly.

• **Input:** sides (3,4,5), (5,5,5), (2,2,3), (1,2,3), (1,1,10), (0,4,4)
• **Output:** scalene; equilateral; isosceles; degenerate (1+2=3); impossible; invalid (zero side)
• Solve for the worked instance first. Then ask what would change if one
• number, name, or condition were replaced by something stranger.

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

cs-029 — discuss

• What did the deciding quantity turn out to be?
• Which assumption shaped your answer?
• After the solution is revealed, the class spends five minutes on this prompt: **"What would a 'suite' that can never fail tell you about its tests?"**

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-029 — not embedded in this deck.

• **Input:** ages 11, 12 (Sat), 12 (Mon), 15 (Mon), 16 (Mon)
• **Output:** refused; admitted; refused; admitted; admitted (weekday boundary is 16)
• Solve for the worked instance first. Then ask what would change if one
• number, name, or condition were replaced by something stranger.

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

cs-030 — discuss

• What did the deciding quantity turn out to be?
• Which assumption shaped your answer?
• After the solution is revealed, the class spends five minutes on this prompt: **"Which row in your table would you defend hardest, and why?"**

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-030 — not embedded in this deck.

• **Input:** def fee(x): if x < 3: return 0; elif x <= 5: return 2; else: return 5
• **Output:** black-box misses x=3 exactly (0 vs 2 boundary) and negative x; white-box adds both; the escaped defect is 'x < 3' vs 'x <= 3'
• Solve for the worked instance first. Then ask what would change if one
• number, name, or condition were replaced by something stranger.

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

cs-031 — discuss

• What did the deciding quantity turn out to be?
• Which assumption shaped your answer?
• After the solution is revealed, the class spends five minutes on this prompt: **"What would a 'suite' that can never fail tell you about its tests?"**

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-031 — not embedded in this deck.

• **Input:** randint-based roller; assert exact output fails intermittently
• **Output:** seed for tests; assert invariants (range, distribution over 10k trials)
• Solve for the worked instance first. Then ask what would change if one
• number, name, or condition were replaced by something stranger.

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

cs-032 — discuss

• What did the deciding quantity turn out to be?
• Which assumption shaped your answer?
• After the solution is revealed, the class spends five minutes on this prompt: **"Which row in your table would you defend hardest, and why?"**

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-032 — not embedded in this deck.

Common misconceptions

• Testing only the happy path - boundaries are where defects live.
• One random outcome deciding a random function's fate.
• Full line coverage mistaken for correctness.
• Adversarial cases dismissed as 'unrealistic' - users are adversarial by nature.

Summary & exit question

• Named technique today: Testing Before Trusting
• Four cases attempted — methods before answers

Exit question (choose one, 60 seconds)

1. Name the three test-case classes and one example of each.
2. Black-box vs white-box in one sentence each.
3. What makes a randomness test fair?