How a parity harness proves a system rewrite

A rewrite is ready when it behaves like the system it replaces under the inputs that matter. A clean architecture, passing unit tests, and familiar screens do not prove that the new system posts the same invoice totals, chooses the same tax basis, or rejects the same malformed adjustment. A parity harness does. It sends the same recorded request to both systems, captures every observable result, normalizes only the fields that are allowed to vary, and produces a difference that an owner can explain.

That sounds like ordinary regression testing until money, dates, state, and side effects enter the room. Then the easy comparison breaks. The old application may read the clock halfway through a calculation, depend on record order, round after each line, reuse a stale exchange rate, or write five rows before returning one response. Some of those behaviors are requirements. Some are accidents that users now depend on. Some are bugs. The harness has to distinguish them without hiding inconvenient evidence.

I use parity as an acceptance argument, not a percentage on a dashboard. The argument has four parts: the replay input represents production, both executions begin from equivalent state, the comparison policy is explicit, and every accepted difference has an owner and a reason. If any part is vague, a green result means very little.

How parity differs from ordinary regression testing

A parity harness compares two implementations of the same behavior; a regression suite compares one implementation with expectations written by people. That distinction changes what each test can discover. A unit test can confirm that a rewritten interest function matches a formula chosen for the test. It cannot tell you that the old system truncates the daily rate before compounding, unless someone already knew to encode that quirk. Replaying the same account through both systems reveals the difference immediately.

The old system acts as an executable specification, but it is not an infallible one. Treating it as an oracle means its output becomes evidence, not truth. The harness should report that legacy returned 104.17 and rewrite returned 104.18. A product owner, finance controller, or named domain owner decides whether the cent belongs to the contract or to a defect. The test machinery must not make that policy decision by silently rounding both values until they match.

Regression tests still matter. They isolate rules, cover synthetic boundaries, and remain fast enough for every commit. Parity covers the combinations that nobody remembered to describe: a line with a zero value after a credit memo, a customer with two billing calendars, or an RPG program that treats blanks differently from zeroes. Keep both. Once a difference is adjudicated, turn the decision into a focused regression test so the team does not have to rediscover it from another large replay.

The harness also compares more than the visible response. For a transaction, the observable contract may include database mutations, emitted files, queue messages, ledger entries, status codes, error categories, and ordering. If the replacement returns the right total but posts it to the wrong accounting period, parity limited to the response gives a dangerous pass. Define the observation boundary before recording traffic, and include every effect that another system or human can see.

Recorded traffic needs a replay contract

A useful recording contains enough information to reproduce intent without copying an uncontrolled heap of logs. Raw access logs rarely qualify. They may omit request bodies, authenticated identity, session state, headers that select a business rule, or the sequence that created the current database state. They can also contain secrets that should never enter a test store.

Use a versioned replay envelope. The envelope says what arrived, which context affected behavior, what state checkpoint the case expects, and which outputs the harness will inspect. One practical shape is:

{
  "case_id": "close-004812",
  "captured_at": "2026-01-31T23:58:42Z",
  "operation": "POST /accounts/close",
  "identity_ref": "role:month_end_operator",
  "state_checkpoint": "ledger-2026-01-31-r7",
  "request": {"account_id": "A1842", "period": "2026-01"},
  "observe": ["response", "ledger_entries", "outbox"]
}

Store references to identities and secrets, never live credentials. Replace personal fields only through a deterministic mapping, because random anonymization can destroy joins and business meaning. If customer 842 appears in a request, a ledger row, and an address table, the sanitized fixture must keep those references connected. Preserve string lengths and character classes when validation depends on them.

Sampling deserves a written policy. A random slice of common requests gives volume but often misses the rare cases that carry financial or operational risk. Combine a production sample with deliberate strata: operation type, success and failure, high and zero amounts, calendar boundaries, unusual encodings, jobs that run for a long time, and every branch tied to money or entitlement. Record sessions with multiple steps as ordered cases when later calls depend on earlier ones.

Keep the original capture immutable. Derive sanitized and replayable fixtures from it with versioned transforms, and record the transform version in every run. Otherwise a changed scrubber can alter the input while the team thinks it is evaluating a changed program. Access to captures should be narrow, retention should be finite, and a failed case should reveal identifiers through controlled diagnostics rather than dumping full records into CI logs.

Equivalent starting state is part of the test

The same request against different data proves nothing. Before each replay unit, both implementations need logically equivalent state, including reference tables, feature settings, sequence positions, batch cutoffs, and records created by earlier steps. This is usually harder than sending the request.

Choose the smallest reset boundary that preserves meaning. A pure quote calculation may run from a compact fixture per case. An account close may require an ordered scenario with several prior postings. A nightly batch may need a full database checkpoint and a controlled queue. Resetting the whole environment before every request is slow, while reusing one mutable database lets cases contaminate each other. Group cases into independent scenarios, reset at scenario boundaries, and keep request order within each group.

Do not require identical physical schemas. A modernized service and Postgres target should not mimic VSAM files or an AS/400 physical file merely to make comparison convenient. Build state adapters that express the same business facts in each representation. Then compare observable business results, not internal table layouts. Architecture parity would defeat the purpose of the rewrite.

State setup must include values that teams tend to forget: the business date, timezone database, locale, currency metadata, rounding mode, tax tables, exchange rates, sequence seeds, and authorization context. Pin these dependencies to a fixture version. If the legacy job reads a rate table by effective date and the rewrite uses today's latest row, identical request JSON still drives different problems.

Verify setup instead of trusting the loader. Before execution, calculate a compact semantic fingerprint such as counts and hashes over canonical business records. The old and new fingerprints will not be identical at the byte level across schemas, but they can assert facts such as 418 open items, the same summed principal by currency, and the same set of active rule identifiers. A setup failure should stop the case as invalid, not appear later as an application mismatch.

Exact comparison starts with canonical values

Compare money as decimal values with declared scale and currency, never as formatted strings or binary floating-point approximations. The phrase "to the cent" sounds simple, but teams routinely compare 12.3 with "12.30", convert both through a floating type, or round only at the final total when the old program rounds each line. The harness needs the domain's arithmetic contract.

IEEE 754 explains why binary floating-point cannot exactly represent many decimal fractions. That does not make every floating calculation wrong; it means an unexplained epsilon is a poor policy for financial parity. Parse monetary outputs into decimal coefficients and scales, then apply the business rule at the same stage as the production system. If the contract says round each tax line half away from zero, do that before summing. If the contract says preserve four decimal places in an intermediate rate, compare that intermediate value when it is observable or add a focused diagnostic.

Canonicalization should remove representational noise while preserving meaning. It can normalize a timestamp to UTC, sort an explicitly unordered set by stable business fields, treat absent optional JSON fields according to the API contract, and decode a padded value of fixed width. It must not lowercase identifiers whose case matters, sort a sequence whose order users see, discard duplicate rows, or turn every error into a generic failure.

Make the policy readable as data rather than burying it in comparator code:

rules:
  - path: response.total_due
    type: decimal
    scale: 2
    tolerance: 0
  - path: response.generated_at
    type: timestamp
    mode: injected-clock
  - path: outbox[*].headers.trace_id
    mode: ignore
    reason: generated transport identifier
  - path: response.allocations
    mode: ordered

That policy prevents an innocent cleanup from widening tolerance across the suite. Require a reason for every ignored path and every nonzero tolerance. Review policy changes like production code, because a comparator can erase a defect more efficiently than any rewrite can create one.

Report differences at the business field, not as two giant JSON blobs. A useful result says invoice.lines[7].tax: legacy 1.34, rewrite 1.35, followed by the applicable comparison rule and fixture version. Include aggregate counts, but never let a 99.9 percent match rate conceal which tenth failed. One wrong payroll deduction outweighs thousands of identical health checks.

Nondeterminism should be controlled before it is ignored

Most nondeterminism can be turned into input. Inject a clock, seed the random generator, reserve identifiers, freeze reference data, and isolate concurrency. When both programs consume the same explicit values, the harness tests behavior instead of comparing two accidents.

Classify every varying field into one of four groups. Controlled values receive the same input in both systems. Canonical values differ in representation but reduce to the same meaning. Ignored values have no business meaning, such as a transport trace identifier. Statistical outputs require a separate test because one replay cannot prove parity for a stochastic model. This classification is sharper than a global ignore list and gives reviewers something they can challenge.

Time causes the most avoidable failures. A process can read the clock at request receipt, at posting, and again while formatting a response. Merely overwriting the final timestamp leaves logic at date boundaries uncontrolled. Route every business clock read through an injectable source in the rewrite. For the legacy side, run in an isolated environment with a controlled system time when safe, intercept its time provider, or capture the values it used and compare derived business outcomes within a scenario that cannot cross a boundary. Never shift a shared production host's clock.

Generated identifiers need correlation rather than equality when they carry no domain meaning. Suppose both systems create a new claim ID, then use it in three rows and an event. Map the legacy ID to the rewrite ID at the creation point and verify that all later references preserve the same relationship. Ignoring every ID field would miss a broken foreign reference. Requiring equal sequences would couple the rewrite to an implementation detail.

Concurrency needs repeated, scheduled tests rather than wishful normalization. If result order is contractually irrelevant, compare a multiset and still check multiplicity. If two postings race for the same balance, force both interleavings with barriers around the contested read and write. A broad tolerance cannot excuse lost updates. Keep load and soak testing separate from semantic parity, but feed any failure found there back into a small reproducible parity scenario.

Side effects require capture, not duplication

A replay must not send real payments, emails, print jobs, or partner messages. Replace the boundary with a recorder that accepts the same command, returns a controlled acknowledgement, and stores a canonical representation for comparison. The point is to prove that the new system intended the same effect, not to perform the effect twice.

Place recorders at the last owned boundary. Capturing an internal function call can pass even when serialization, routing, or headers are wrong. Capturing beyond the boundary can affect a third party. For a message broker, record the final topic, business headers, partitioning field when it has meaning, and payload after serialization. For a file interface, capture bytes as well as a parsed business view when fixed widths, encodings, or line endings form part of the contract.

Database effects need comparison that understands transactions. Take a before snapshot of the relevant business entities, run the case, then calculate an after delta. Compare inserts, updates, deletes, and invariants across the two representations. Do not compare audit timestamps or surrogate IDs directly unless consumers rely on them. Do compare whether debits balance credits, whether one outbox row exists per committed action, and whether rollback leaves no partial business change.

Failures are outputs too. Match the class of failure, status, recoverability, and side effects. Exact legacy error prose may not deserve preservation, especially if the rewrite returns a structured error, but callers may depend on an error code or retry signal. Write that compatibility rule down. A replacement that converts a permanent validation failure into a retryable server error can cause more damage than a display difference of one cent.

The recorder should expose duplicate attempts. During retries, compare idempotency behavior across the full sequence: first request, lost acknowledgement, repeated request, and final state. Seeing two identical outbound commands is not harmless merely because a downstream sandbox accepted both.

A mismatch needs a decision trail

Every mismatch should enter a small, explicit workflow: reproduce it, localize it, classify it, assign an owner, and encode the decision. Without that trail, teams either chase harmless timestamps for days or wave away financial differences to protect a deadline.

Start by shrinking the failing replay. Preserve its state checkpoint, remove unrelated records, and reduce the request sequence until the difference remains. Then inspect the earliest divergent observable value, not the last total. A discrepancy of one cent on an invoice might begin with a rounding difference in one line; twenty later fields merely repeat it. Store the minimized case beside the harness so it runs on every change.

Use a short set of classifications:

• Rewrite defect: the new implementation violates accepted legacy behavior.
• Harness defect: state, capture, normalization, or observation is wrong.
• Approved correction: the legacy behavior is wrong and an owner authorizes a new result.
• Contract change: the organization intentionally changes behavior beyond a defect repair.
• Unresolved: evidence or ownership is still missing, so release remains blocked for that case.

An approval record should name the case, field, legacy value, new value, rationale, approver, decision date, and the regression test that now defines expected behavior. Avoid vague waivers such as "rounding issue accepted". Six months later, nobody can tell whether that waiver covered one tax jurisdiction or all calculations. Expire broad exceptions and forbid wildcard ignores without a bounded reason.

A useful failure artifact fits in a review without exposing the whole production record. Include sanitized inputs, semantic state fingerprint, both normalized outputs, a structured diff, policy version, and replay command. That package lets an engineer reproduce the result while a domain owner reviews the actual business consequence.

When the old system is wrong, preserve evidence rather than behavior

Known legacy defects should become approved divergences, never comparator tricks. First prove that the rewrite differs. Then prove why the old result violates the chosen rule. Finally record who owns the decision to change observable behavior. This keeps technical migration authority separate from business authority.

The popular advice to "match first, fix later" is useful only when later is real. It reduces simultaneous variables and makes migration easier to reason about. It is wrong when it ships a known overcharge, recreates an unsafe authorization path, or entrenches corrupt data because nobody scheduled the second change. Severity and reversibility decide the order. Preserve harmless oddities temporarily if that cuts release risk. Correct harmful behavior before cutover with explicit approval and communication.

Run approved corrections through two assertions. The parity assertion documents the intentional difference: legacy produces X, replacement produces Y. The business regression assertion proves Y from an independent rule or example. If you delete the legacy side later, the second test survives as the durable specification. This also stops a future maintainer from "fixing" the rewrite back to the old defect after seeing a red parity case.

Historical data may carry the defect forward. Correct code can still disagree with old reports because stored balances, status flags, or derived fields already contain bad results. Decide whether to migrate, recompute, quarantine, or preserve each affected record class. Rehearse that data policy in the same checkpoint used for replay. Code parity without data disposition leaves the cutover argument incomplete.

Communicate changed behavior at the boundary where users or dependent systems notice it. A corrected amount may require a reconciliation report. A stricter validation may expose records that the old system silently accepted. An authorization correction may invalidate a workflow. The decision trail should state the operational response, not just the arithmetic rationale.

The legacy runner needs a stable interface

The harness should invoke the old system through the narrowest stable boundary that still exercises real behavior. An HTTP endpoint is convenient when it already exists, but many legacy paths begin with a batch file, a queue record, a terminal transaction, or a stored procedure. Wrap that entry point with a runner that accepts the replay envelope, establishes context, waits for completion, and returns captured observations in one versioned result format. Do not rewrite business logic inside the wrapper. Every rule copied there creates another implementation that can disagree.

Treat runner health separately from application output. A timeout, unavailable region, failed fixture load, or recorder fault makes the case inconclusive. It does not mean the legacy system returned an error, and it certainly does not count as parity. Use a result envelope that distinguishes completed, application_failure, and infrastructure_failure, then attach job logs or diagnostic codes under controlled retention. This distinction keeps unstable test plumbing from inflating the rewrite's apparent match rate.

Resource isolation matters when the old platform has global state. Two parallel replays may share temporary files, job names, sequence generators, or a work table that production code assumes has one writer. Put explicit locks around those resources or allocate isolated namespaces when the platform permits it. If isolation is impossible, serialize the affected scenario and say so in the suite metadata. A fast harness that changes the behavior it measures gives worse evidence than a slower honest one.

Version the runner, adapters, comparison policy, fixtures, and candidate build in each result. A replay identifier should be enough to reconstruct all five. Keep execution artifacts addressed by content where possible so a reviewer can prove that a later report uses the same normalized outputs. Protect final acceptance bundles according to the organization's existing change controls. The harness does not need a new bureaucracy, but its evidence should be at least as durable as the release approval it supports.

Finally, test the harness against planted differences. Change one cent, drop an outbox record, swap two ordered allocations, shift the business date, and force a rollback leak in a controlled fixture. Each mutation should produce the expected failure at the expected field. Teams test application code constantly and assume the comparator works. A comparator that has never demonstrated its own sensitivity is an untested part of the migration.

Release evidence must be harder to game than a pass rate

A credible release gate names the covered behavior and the remaining uncertainty. It does not say only that 98 percent of cases passed. Report coverage by operation and risk class, the number of exact matches, approved corrections, harness failures, unresolved differences, and untested boundaries. Show whether the sample includes period close, rollback, retry, malformed input, transactions with high values, and the oldest supported data shapes.

Keep the gate strict: no unresolved difference in a class that is critical for release, no unexpected side effect, no comparison policy change hidden inside the candidate build, and no fixture setup failure counted as a pass. Mismatches with lower risk can follow a documented risk decision, but they still appear in the evidence. A denominator that silently excludes crashed replays is fraud by spreadsheet.

Run the same corpus more than once. Repeatability detects uncontrolled time, order, shared state, and environmental leakage. Then replay a recent holdout capture that developers did not use while tuning the rewrite. A corpus can become an overfit target just like a unit suite. The holdout does not need to be enormous; it needs representative, protected provenance and a documented sampling method.

At cutover, shadowing can add evidence if the system allows it safely. Send a copy of eligible live reads or commands to the replacement, suppress its effects, and compare results off the user path. Never shadow an operation by letting both sides commit. Monitor privacy, capacity, and timing changes introduced by the shadow path itself.

CodeHero uses recorded production traffic and a parity harness to hold rewritten Go, Rust, and TypeScript systems to original behavior while changing their architecture. For a project delivered in under 30 days, that evidence has to be designed at intake, not assembled at the release meeting.

The acceptance bundle should also expose what the replay did not prove. List operations with no usable capture, integrations represented only by a stub, data vintages absent from the fixtures, and concurrency patterns tested only under load. For each gap, name the compensating evidence, such as a focused regression suite, a reconciliation query, a staged operator rehearsal, or a observation for a limited period after cutover. Do not convert those controls into parity claims. They answer a different question and should stay labeled as such. The release approver can then judge a bounded risk instead of assuming that a large case count covers every path. Keep this gap register attached to the final result, because it becomes the first test plan for production monitoring and the next capture cycle. If an unrepresented operation appears during shadowing, add it to the corpus with its provenance intact. Evidence improves when new cases extend a declared boundary. It becomes less trustworthy when the team quietly changes what the word coverage means.

A parity harness earns trust through the differences it refuses to hide. Make inputs reproducible, state equivalent, arithmetic explicit, varying fields classified, and exceptions owned. Then the last red case is useful evidence, not an obstacle to paint green.