Problem Solving — CS & DS

Lecture 27 — Tables, Cleaning, and Anomalies

Learning objectives

Case-based lecture: you attempt, we discuss, then we reveal.

Today

The plan

Key definitions

Key definitions

Explain each in one breath; the cases will do the heavy lifting.
Worked example

The Room Booking Audit: rule-based cleaning

cs-105

Bookings with issues: '9:00-10:00' (fine), ' 10:30 - 11:30 ' (padded), '24:00-25:00' (invalid), '9:00-9:00' (empty). Write rules, apply them, report what was dropped and why.

raw = ["9:00-10:00", " 10:30 - 11:30 ", "24:00-25:00", "9:00-9:00"]
kept, dropped = [], []
for r in raw:
    norm = " ".join(r.split()).replace(" ", "")
    s, e = norm.split("-")
    sh, sm = map(int, s.split(":"))
    eh, em = map(int, e.split(":"))
    start, end = sh * 60 + sm, eh * 60 + em
    if not (0 <= start < end < 24 * 60):
        dropped.append((r, "invalid range"))
    else:
        kept.append(f"{start:04d}-{end:04d}")
print(kept)
print(dropped)

Expected: kept: 9:00-10:00 and 10:30-11:30 (normalised); dropped: the invalid and empty ranges

Cost: O(n) rows, each parsed in constant time.

Worked example

The Grade Typos: anomaly detection with limits

cs-106

Scores 78, 92, 850, 65, 88, -5. Flag impossible values; then discuss what you can and cannot conclude about suspicious-but-possible ones (e.g. 92 when the class mean is 40).

scores = [78, 92, 850, 65, 88, -5]
valid = [s for s in scores if 0 <= s <= 100]
mean = sum(valid) / len(valid)
var = sum((s - mean) ** 2 for s in valid) / len(valid)
sd = var ** 0.5
review = [s for s in valid if abs(s - mean) / sd > 2]
print(valid, mean, review)

Expected: hard-dropped: 850 and -5; statistical review: none after cleaning

Cost: O(n); the guard against sd = 0 is where most student implementations break.

Worked example

The Sensor Dropout: missing values as a decision, not an accident

cs-108

Hourly readings 12.0, 11.5, None, None, 13.0, 12.5. Choose a policy (drop rows, fill zero, carry forward, fill mean), apply it, and state how the choice changes the story.

readings = [12.0, 11.5, None, None, 13.0, 12.5]
present = [r for r in readings if r is not None]
drop_mean = sum(present) / len(present)
zero_fill = [r if r is not None else 0 for r in readings]
fwd = []
last = present[0]
for r in readings:
    if r is not None:
        last = r
    fwd.append(last)
mean_fill = [r if r is not None else drop_mean for r in readings]
print(len(present), round(drop_mean, 2))
print(round(sum(zero_fill) / len(zero_fill), 2), round(sum(fwd) / len(fwd), 2), round(sum(mean_fill) / len(mean_fill), 2))

Expected: drop: mean 12.25 (n=4); zero: 8.17; forward: 12.33; mean: 12.25

Cost: O(n) per policy; the cost is in the argument, not the loop.

Case attempt · lecture 27
Case cs-105 · Beginner

The Room Booking Audit

Booking rows with missing rooms, end-before-start times, and double bookings. Define rules, count violations, decide repairs vs flags.

⏱ 5 minutes — attempt now

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

Discussion

cs-105 — discuss

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-105 — not embedded in this deck.

Case attempt · lecture 27
Case cs-106 · Foundational

The Grade Typos

Scores like 105, -3, 'eighty'. Categorise defects, choose a policy per category (fix, flag, drop), and defend each choice.

⏱ 5 minutes — attempt now

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

Discussion

cs-106 — discuss

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-106 — not embedded in this deck.

Case attempt · lecture 27
Case cs-107 · Intermediate

The Duplicate Students

Same student appears with name variants and id typos. Design a matching rule with a manual-review queue; measure how much the rule catches vs leaves.

⏱ 5 minutes — attempt now

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

Discussion

cs-107 — discuss

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-107 — not embedded in this deck.

Case attempt · lecture 27
Case cs-140 · Intermediate

The Sliced Anomaly

Two campus bookshops, each with a coffee bar. Per-shop revenue per customer:

⏱ 5 minutes — attempt now

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

Discussion

cs-140 — discuss

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-140 — not embedded in this deck.

Case attempt · lecture 27
Case cs-108 · Advanced

The Sensor Dropout

Temperature sensor with gaps and spikes. Distinguish dropout vs anomaly, impute or flag, and show how each choice changes a downstream average.

⏱ 5 minutes — attempt now

Pseudocode, flowchart or Python. State one assumption. Compare with your neighbour at minute 4.

Discussion

cs-108 — discuss

Instructor: the reveal, the reasoning section and common wrong turns are in the instructor distribution for cs-108 — not embedded in this deck.

Check yourself

Common misconceptions

Summary

Summary & exit question

Exit question (choose one, 60 seconds)

  1. Why report every cleaning rule?
  2. What is a false merge in cs-107?
  3. When is a gap an anomaly?
Expected answers (instructor): silence is falsification of the analysis; the tally is the audit trail two different students collapsed by a weak matching key when it breaks the series' established rhythm — context decides
→ / space: next · ←: back · Home/End: jump · Ctrl+P → PDF