Before a customer’s data can flow into your system, it has to be mapped: each of their columns pointed at one of your canonical fields, and each value coerced to a type. Their age_txt becomes your age: int; their status becomes your active: bool. Get the mapping right and the clean rows validate while the genuinely bad ones are quarantined — held aside, not silently dropped. Get it wrong — point age at the name column — and everything fails at once. Pick the mapping for each field and watch validation light up, row by row. This is the boundary discipline from the integration handbook, and the CSV mapper challenge, made visual.
map each column to a field · coerce to a type · quarantine bad rows (don’t drop) · wrong mapping fails everything
canonical schema
The clean, typed shape your system expects — the target every customer export must be mapped onto.
coercion
Converting a raw string value to a type: "34" → 34 (int), "true" → true (bool). It fails on bad values.
quarantine
Setting aside rows that don’t coerce, for inspection — instead of silently dropping them and losing data.
mapping
The choice of which customer column feeds each canonical field. The one decision that makes or breaks the import.
mapping.js — coerce at the boundary
Fix the mapping
Three canonical fields need a source column. The starting mapping is wrong — watch every row fail. Click each field to cycle which customer column feeds it. When the mapping is right, the clean rows go green and only the genuinely bad values are quarantined.
0
rows valid
4
quarantined
✗
mapping
How it works
The lab is a tiny mapping-and-validation engine, and building it teaches why mapping is the highest-leverage — and most dangerous — step of a data import. A canonical schema names the fields you want and their types. A mapping assigns each field a source column from the customer’s export. Validation then walks every row and tries to coerce the mapped value to the field’s type: a string like "34" becomes the integer 34, "true" becomes the boolean true, and anything that can’t convert — "N/A" as an int, "yes" as a strict bool — raises an error. A row is valid only if all of its fields coerce; otherwise the whole row is quarantined, set aside for a human to inspect rather than silently dropped, because silently dropping rows is how an import corrupts a dataset without anyone noticing. Two lessons fall out immediately. First, a wrong mapping is catastrophic and total: point the integer age field at the name column and every single row fails coercion at once — the failure isn’t subtle, which is actually a gift, because it’s obvious. Second, even a correct mapping doesn’t make every row valid: some values are genuinely bad (a missing age, a status of "yes" where you expected true/false), and the right behavior is to quarantine exactly those rows and let the good ones through. That distinction — a broken mapping versus a few broken rows — is the one an FDE has to read at a glance, because the fixes are completely different: re-map the field, versus hand the quarantine list back to the customer.
1
A schema wants typed fields
Your system expects age as an int, name as a string, active as a bool. The customer’s export has differently-named columns of raw strings.
2
Mapping assigns a source column
Each canonical field is pointed at one of the customer’s columns. This single choice determines whether the import works at all.
3
Coercion validates row by row
Each mapped value is coerced to its field’s type. A row is valid only if all its fields coerce; otherwise it’s quarantined for inspection — never silently dropped.
✓
Read broken-mapping vs broken-rows
A wrong mapping fails every row at once (re-map the field). A correct mapping still quarantines the genuinely bad values (hand them back to the customer). Different symptoms, different fixes.
Wrong mapping
all rows fail
Right mapping
bad rows quarantined
Never
silently drop rows
Coerce at
the boundary
The code
# Map + coerce a customer export onto a canonical schema
schema = {"age": "int", "name": "str", "active": "bool"}
mapping = {"age": "age_txt", "name": "name", "active": "status"} # the key decisionfor row in customer_rows:
try:
clean = {f: coerce(row[mapping[f]], t) for f, t in schema.items()}
valid.append(clean)
except BadValue:
quarantine.append(row) # set aside — do NOT drop silently# wrong mapping → every row raises. a few bad values → a few quarantined.
Quick check
1. You import a customer export and every single row fails validation. What’s the most likely cause?
When all rows fail at once, the problem is almost never the data — it’s the mapping. A typed field pointed at the wrong column (an int field at the name column) fails coercion on every row. The fix is to re-map, not to clean the data.
2. With a correct mapping, a few rows still fail. What should you do with them?
Some values are genuinely bad (a missing age, an unexpected status). The right behavior is to quarantine those specific rows — set them aside, surface them — not to drop them silently (which loses data invisibly) or force garbage through (which corrupts your dataset).
3. Why coerce and validate data at the boundary of your system?
Validating at the boundary means a malformed value is rejected or quarantined at the point of entry, before it flows into your database, models, or reports. Let it in unchecked and a single bad row can silently corrupt everything that trusts the data downstream.
FAQ
What is data mapping in an integration?
Matching a source system’s fields to your target’s fields, including converting values to the right types. A customer export might have "age_txt" holding "34"; your system wants "age" holding the integer 34. Mapping decides which source column feeds each target field, and coercion converts the value — the critical step that lets messy data flow into a clean, typed system.
What should you do with rows that fail validation?
Quarantine them — set them aside for inspection — not silently drop them or force them through with defaults. Silent dropping loses data invisibly; forcing garbage corrupts the dataset. Quarantining makes bad rows visible: count them, show the customer which records failed and why, and fix the source or mapping. A good quarantine report often builds the customer’s trust in the import.
Why does a wrong mapping fail every row?
Because a mapping error is structural, not per-row: point an integer field at a names column and every value is a name, so every coercion to int fails — all rows at once. That’s useful — total failure obviously signals a wrong mapping, versus a few scattered failures signaling genuinely bad data. Reading "all failed → fix the mapping" versus "some failed → quarantine and inspect" is a core skill.
What is schema drift and how do you handle it?
Schema drift is when a customer’s data shape changes over time — a renamed column, a new field, a format change — silently breaking a working integration. Handle it by validating at the boundary (drift shows up as a spike in quarantined rows, not silent corruption), alerting when the quarantine rate jumps, and keeping the mapping easy to update. Never assume this week’s file matches last week’s; validation and quarantine catch drift before it does damage.