The data quality problem underneath every automation
A merge field pulling the wrong first name used to embarrass one person at a time. Automated, it embarrasses four thousand at once, on a schedule, with your logo at the top. Automation does not create data problems and it does not fix them, it converts private ones into public ones. Before adding sequences, it is worth knowing precisely which defects your database has, because a handful of them account for nearly all the visible failures.
Which defects actually cause visible failures?
Five, in rough order of how often they end up in front of a customer. Duplicate records, so one person receives the same sequence twice from two slightly different email addresses. Empty personalisation fields, where a missing first name renders as a blank or the word null. Stale employment, where the contact left the company and the address now bounces or reaches a shared inbox. Consent state held in two systems that disagree. And lifecycle stage written by two processes, so a customer re-enters a prospect sequence.
Notice that none of these are exotic. They are all ordinary consequences of collecting data from several sources over several years without anyone owning the result, which describes almost every database.
The reason they matter more under automation is timing. A person composing an email sees a blank name field and fixes it. A scheduled send does not look, and the failure reaches everyone in the segment simultaneously.
Why do duplicates keep coming back?
Because the systems involved disagree about what makes two records the same person. Most marketing platforms treat the email address as the identity, so one human with a work address and a personal one is two people. Most CRMs treat the record identifier as identity and allow two records with the same email. Sync those two together and each system faithfully recreates the other's duplicates.
Form design generates the rest. A form that creates a new record on every submission rather than matching on email will produce a duplicate every time someone downloads a second document, and the second record often carries fewer fields than the first, so a merge overwrites good data with blanks.
The fix is to pick one identity key, decide which system owns it, and configure every form and integration to match on it before creating. Deduplication runs are cleanup, not prevention, and a database that needs one every quarter has an intake problem rather than a hygiene problem.
How fast does contact data go stale?
Faster than most retention policies assume, because job changes, company changes and address changes all invalidate a record independently. A business email list that has not been touched for a year will contain a meaningful share of addresses that no longer reach the person, and there is no way to know which ones without sending.
The practical consequence is that a large dormant list is a deliverability liability rather than an asset. Sending to it after a long gap produces bounces and complaints in a concentrated burst, which is exactly the pattern that harms sending reputation.
So decay should be handled by policy rather than by hope. Define what makes a record dormant, define what happens when it becomes dormant, and apply it automatically. The alternative is a database that grows forever and gets less useful every month while looking, on the dashboard, like an asset.
| Defect | How it shows up | Root cause | Prevention |
|---|---|---|---|
| Duplicate contacts | One person gets the same sequence twice | Two systems disagree about the identity key | One key, one owning system, forms match before creating |
| Empty merge fields | Blank or null in the greeting line | Optional form fields used in required personalisation | Default values, and a hard rule that merge fields must have fallbacks |
| Stale employment | Bounces, or replies from a colleague | No re-verification and no dormancy policy | Suppress after a defined period of no engagement |
| Conflicting consent state | Someone who unsubscribed receives mail | Opt-out stored in both the CRM and the platform | One system owns consent, the other reads it only |
| Lifecycle stage overwritten | A customer receives a prospecting sequence | Two processes writing the same field in both directions | Field-level ownership, with the field read-only on the other side |
| Free-text job titles | Segments and fit scoring miss obvious buyers | Open text capture with no mapping | Map to a picklist on entry, keep the raw string separately |
Which fields deserve real discipline?
Very few, and choosing them is the whole discipline. Email address, consent state, lifecycle stage, owner, and whichever two or three fields your segmentation genuinely relies on. Those need a defined format, a single owning system, and validation on entry. Everything else can be messy without consequence.
This is worth stating because data quality programmes usually fail by being comprehensive. A project to clean every field in a database of a hundred thousand records never finishes and never gets used. A project to make five fields trustworthy finishes and changes what the team can safely automate.
Job title is the field most worth mapping, because fit scoring and segmentation both depend on it and free-text capture makes both unreliable. Store the raw string, map it to a controlled list on entry, and segment on the mapped value.
What audit can you run this week?
Six queries against your contact database, none of which need a tool. Count records sharing an email address. Count records where a field used in any live merge tag is empty. Count records with no engagement at all in the last twelve months. Count records whose consent state differs between your CRM and your sending platform. Count distinct values in your job title field. And count records created in the last quarter with no source recorded.
Each number is a specific decision rather than a general worry. A large empty-merge-field count tells you to add fallbacks before the next send. A large consent mismatch tells you to stop sending until one system is made authoritative, because that one has legal consequences rather than aesthetic ones.
Run the same six queries a quarter later. Improvement in the counts is the only evidence that a hygiene effort worked, and it costs an hour to produce.
Should you clean before automating or after?
Clean the fields the first sequence touches, and nothing else. Waiting for a clean database before launching anything is a familiar way to spend a year, and the database will not be clean at the end of it because the intake problems that caused the mess were never addressed.
The order that works is narrow and repeatable: pick the sequence, list the fields it reads, fix those fields for the segment it targets, launch, then fix the intake path so those fields stay correct for new records. That last step is what stops the work being repeated.
There is one exception. Consent state has to be correct before any send, for the whole audience rather than the segment, because the cost of getting it wrong is regulatory rather than cosmetic. That is the one clean-up that genuinely blocks launch.
Common questions
- Why does marketing automation make data quality problems worse?
- It does not create them, it publishes them. A person writing an email notices a blank name field and fixes it before sending. A scheduled sequence does not look, so the same defect reaches the entire segment at once, repeatedly, on a schedule. Automation converts small private data errors into visible ones, which is why a handful of specific fields need to be trustworthy before any sequence goes live.
- Why do duplicate contacts keep reappearing after a clean-up?
- Because the connected systems disagree about what identifies a person. Marketing platforms usually treat the email address as identity, while CRMs treat the record identifier as identity and permit two records sharing an address. Synced together, each faithfully recreates the other's duplicates. Deduplication is cleanup, not prevention. Choose one identity key, decide which system owns it, and make every form match before it creates.
- Which contact fields actually need to be clean?
- Email address, consent state, lifecycle stage, record owner, and the two or three fields your segmentation genuinely relies on. Data quality programmes usually fail by trying to be comprehensive across every field in a large database. A narrow effort that makes five fields trustworthy finishes, and it changes what the team can safely automate. Everything else can stay messy without consequence.
- How often does business contact data go out of date?
- Quickly, because job changes, company changes and address changes each invalidate a record independently, and there is no way to know which addresses have failed without sending to them. A list left untouched for a year will contain a meaningful share of unreachable addresses. Sending to a long-dormant list produces bounces and complaints in a concentrated burst, which is the pattern most likely to damage sending reputation.
- Should you clean your database before setting up automation?
- Only the fields the first sequence reads, for the segment it targets. Waiting for a fully clean database delays launch by a year and does not work, because the intake paths that created the mess remain. The exception is consent state, which must be correct across the whole audience before any send, since the consequence of getting that wrong is regulatory rather than cosmetic.
