How AI coding tools change code you did not write

Autocomplete, assistants, and agents can all produce a plausible function. That superficial similarity has encouraged vendors and engineering teams to use one label for three different operating models. The label hides the part that matters: what the tool can observe, what it can change, and what evidence it collects before declaring the work done.

On a codebase you know, those differences affect speed. On a system whose authors are gone, they affect whether you preserve behavior that nobody remembered to write down. A completion model predicts text near the cursor. An assistant responds to a bounded body of context. An agent chooses actions, observes their results, and keeps going. Treating one as another produces false confidence long before it produces a compiler error.

How autocomplete predicts the next local edit

Autocomplete proposes text from the code around the cursor and whatever additional repository context its host supplies. Its natural unit of work is the next token, line, or small block. Even when a completion spans a function, the interaction remains predictive: the tool offers text, and the developer decides whether to accept it.

This model is excellent when intent already lives in the current file. Repeating an error-handling pattern, filling a switch over a known enum, building a test table from adjacent cases, or writing another method that follows a visible interface all fit. The developer holds the design in their head. The tool removes typing and recalls syntax.

The Language Server Protocol makes a useful distinction here. Its completion request carries a document position and can carry a trigger character or trigger kind. That protocol does not define a mission such as "replace this storage layer" or an evidence loop for proving the replacement. A product may enrich completion with indexed repository snippets, but the interaction still ends at a suggestion. More retrieval can improve the guess without turning the guess into an investigation.

Autocomplete stops being useful when correctness depends on facts outside the supplied neighborhood. A COBOL paragraph may update a field whose layout comes from a copybook, while JCL selects a different input dataset at month-end. A VB6 event handler may look unused until a form file binds it by name. A PL/SQL procedure may rely on a package variable initialized by an earlier call. The missing fact often has valid syntax and no visible marker at the edit point.

Watch the acceptance gesture. Pressing Tab means "insert these characters," not "I have established that this behavior is safe." Teams get into trouble when the physical ease of acceptance quietly substitutes for review. A long completion deserves more suspicion, not less, because its polished surface can hide a larger set of assumptions.

How an assistant works inside a supplied conversation

A coding assistant answers a request using the material placed in its context: selected code, open files, retrieved snippets, diagnostics, instructions, and previous turns. Its natural unit is the conversation. It can explain a routine, compare designs, draft a patch, or reason about an error with more room than autocomplete has.

That extra room changes the quality of the work. You can ask an assistant to trace a value across four functions, identify an implicit invariant, or explain why a proposed refactor changes transaction boundaries. You can challenge its first answer and add the file it did not see. This makes it useful for exploration, especially when a developer remains responsible for choosing the evidence.

The boundary is easy to miss because chat interfaces speak fluently about files they have never opened. If the assistant says, "This function is only called by the batch importer," the claim may be a deduction from the pasted snippet rather than a repository search. Ask what it inspected. A sound answer should name the call sites, configuration, generated bindings, or runtime evidence behind the conclusion. If it cannot, treat the statement as a hypothesis.

Context windows do not solve context selection. A million lines can fit badly even when a model accepts a large input. Source trees contain generated code, duplicated versions, database definitions, deployment scripts, binary layouts, and tests with different authority. Putting more tokens into a prompt does not tell the assistant which copybook is active in production or which of two nearly identical calculations is the regulatory one.

Use an assistant to turn uncertainty into explicit questions. Ask it to list the assumptions behind a patch, name files that would falsify those assumptions, and describe the behavioral test it expects to pass. That output gives a reviewer something concrete to inspect. Do not ask for confidence scores. A precise-sounding percentage has no shared calibration and adds nothing to the evidence.

How an agent changes the repository through an action loop

A coding agent can select and execute actions, then use the results to choose its next action. It may search the tree, open files, edit several modules, run a compiler, execute tests, inspect failures, and revise the patch. Its natural unit is a task with a stopping condition, not a single response.

The loop matters more than the chat box. A useful agent does something like this:

1. Resolve the repository state and applicable instructions.
2. Search for definitions, callers, configuration, tests, and generated edges.
3. Make a bounded change on a branch or isolated worktree.
4. Run the checks that can disprove the change.
5. Inspect the diff and report remaining uncertainty.

Each observation can redirect the work. A failed test may expose an undocumented rounding rule. A compiler error may reveal a build tag that selects another implementation. A search may find a second language calling the same database routine. The agent can follow these branches without waiting for a person to paste the next file.

Action does not imply autonomy without limits. File writes, shell commands, network access, credentials, deployment rights, and approval rules are separate capabilities. A tool that can edit the tree but cannot run tests has a shorter evidence loop. A tool that can run arbitrary production commands has a dangerous permission model. Calling both of them "agents" tells an engineering lead almost nothing.

A stopping condition also needs scrutiny. "The tests pass" is adequate only if the tests cover the behavior at risk. "The build succeeds" proves type and linkage constraints, not business equivalence. The agent should stop because it met an explicit acceptance condition and exhausted the agreed checks, not because it ran out of obvious actions or produced a tidy diff.

Unknown systems turn missing context into the main risk

In unfamiliar code, the largest risk is usually not generating the replacement syntax. It is discovering the behavioral contract that crosses source files, job control, data, operations, and user habits. The original authors often encoded that contract in places a modern repository search ranks poorly.

Consider a nightly billing job. A calculation routine reads clearly in isolation, so an assistant rewrites it in a modern language and its unit tests pass. Production behavior also depends on a JCL condition code that skips one step after a partial input, a fixed-width record whose blank amount differs from zero, and an operator rerun procedure that preserves an intermediate file. None of those facts has to appear in the calculation routine. A locally faithful rewrite can still double-charge a rerun.

This is where the field blurs two different kinds of context. Source context is material the tool can read: code, schemas, build files, tickets, and tests. Operational context is evidence of what the system actually does: production requests, batch inputs, outputs, timing, side effects, failure recovery, and operator decisions. More source context cannot automatically recover operational context. The consequence is blunt: repository understanding can support a migration, but it cannot by itself prove behavioral parity.

Legacy systems add cross-language edges. COBOL calls assembler or database procedures. RPG programs depend on CL commands and externally described files. A Classic ASP page invokes COM components built in VB6. An Excel workbook calls an Access query that calls a stored procedure. A tool that indexes only the language named in the ticket will produce a clean map with missing roads.

Before changing such a system, write down the evidence boundary. Which repositories are included? Which schedulers, schemas, data layouts, and runtime traces are available? Which external calls will be simulated? Which operator actions remain outside observation? This is not project paperwork. It tells you which claims the tool can support and which claims still require a person who knows production.

Behavioral parity requires an oracle outside the new code

A rewrite should be judged against observed behavior, not against how reasonable the new implementation looks. The safest oracle is independent of the generated code: recorded inputs sent to both versions, with outputs and side effects compared under explicit normalization rules.

A small parity harness can begin with a manifest that states what counts as equal:

{
  "case": "month_end_partial_input",
  "input": "fixtures/partial.dat",
  "compare": ["stdout", "records", "exit_code"],
  "ignore": ["run_id", "processed_at"]
}

Then run old and new implementations from a clean state and retain machine-readable results:

case                         old  new  records  result
month_end_partial_input      04   04   1827     PASS
blank_amount_field           00   00   19       PASS
operator_rerun_after_step_3  00   08   641      FAIL

The failed row is more useful than a confident code review. It gives the team a reproducible input, the first divergent effect, and a place to investigate. The comparison should cover returned data, database writes, emitted messages, files, exit codes, and ordering wherever consumers observe ordering. Normalize only values that the contract truly permits to vary, such as generated run identifiers. An overbroad ignore list can make every implementation look correct.

Recorded production traffic has gaps. Rare error paths may not appear, sensitive fields may require controlled handling, and batch jobs may depend on clock or environment state. Add designed cases for boundaries, malformed input, retries, and recovery. Keep the old executable available in an isolated harness when licensing and platform access allow it. If no independent oracle exists, say so plainly and reduce the scope of automatic change.

I argue against using model-written tests as the sole acceptance suite. The recommendation is popular because the model can produce code and tests in one pass, and the resulting green build feels complete. Both artifacts can share the same mistaken assumption. Generated tests help express known rules; they do not independently discover the rules the generation missed.

Parity work fails when teams compare only the happy-path return value. State changes often escape through channels the new design intends to remove: a temporary file consumed by another job, a status code checked by JCL, a database row written before an error, or a report sorted in the order operators expect. Inventory observable effects from the outside. If a consumer can distinguish two runs, the harness must either compare that difference or document why the new contract may change it.

Time deserves its own fixture. Old programs commonly read the clock more than once, derive business dates from scheduler variables, or use local midnight as a processing boundary. Freeze time where the platform permits it and record the values supplied by the scheduler. Do the same for random values, sequence generators, locale, encoding, and environment variables. Without controlled inputs, a difference report fills with noise, and reviewers begin ignoring failures that may include the one they need.

Database comparison needs more than dumping final tables. Capture transaction boundaries and failure points when callers can observe them. Two implementations may end with identical rows after success while behaving differently after the third write fails. Run fault cases that interrupt the job at controlled points, then compare committed rows, retry markers, locks, emitted messages, and the result of the documented rerun procedure. This is tedious work, but it turns the vague instruction "preserve behavior" into evidence an engineer can dispute.

Normalization rules should live under review beside the harness. A rule such as "ignore all timestamps" is usually too broad; it can hide a settlement date moved into the wrong period. Prefer field-level rules with a reason, for example ignoring a generated trace identifier while comparing the business-effective timestamp exactly. When a rule changes, rerun prior cases and record which previous differences disappear. The ignore configuration is part of the migration's specification, not housekeeping.

The old system can also disagree with itself. Production recordings may contain an acknowledged defect, an undocumented manual correction, or behavior that varies by deployment. Do not let the agent silently choose the most convenient version. Classify each mismatch as behavior to preserve, a defect to repair under a separate decision, an allowed difference, or an unresolved observation. The owner of the business process must approve the second and third categories. Otherwise a rewrite can smuggle policy changes through a technical review.

A parity result should be reproducible by someone who did not run the original task. Retain the source revision, build inputs, fixture identity, environment description, normalization version, command line, and output artifacts. Hash large or sensitive fixtures when copying them is inappropriate, but keep a controlled path to the originals. Redact data through a defined process rather than editing fixtures casually, because masking can alter field lengths, character sets, checksums, and branch behavior.

Dynamic behavior needs a different discovery method from static call graphs. Search will find a direct function name, but it may miss reflection, string-based dispatch, database callbacks, plugin registration, scheduler invocations, and names assembled from configuration. Combine source search with build metadata and runtime observation. On a system that permits tracing, record loaded modules, executed jobs, called endpoints, opened files, and database routines for representative cases. The trace does not replace source reading; it shows which parts of a broad source map actually participate in the observed behavior.

Coverage claims must name the denominator. Saying an agent "read the repository" can mean it listed every file, embedded selected text, parsed supported languages, or built a cross-language dependency graph. Those are different acts. Ask for counts by file type, explicit exclusions, parse failures, unresolved symbols, and edges inferred from configuration. A short exclusion report is more trustworthy than a sweeping claim of complete understanding.

Finally, test the harness itself with deliberate mutations. Change a comparison field, reverse an ordered output, alter an exit code, and suppress a side effect. Each mutation should produce a clear failure. If the harness stays green, it is not an oracle for that behavior. Teams readily review application code while treating test infrastructure as neutral; in a generated rewrite, the comparison machinery deserves at least the same suspicion as the code it judges.

Autocomplete wins when the developer already owns the intent

Autocomplete is often the best tool for a narrow, understood edit because it keeps latency and ceremony low. If you know the contract, can see the relevant types, and will review each insertion, an action loop adds overhead without finding much new evidence.

Good completion work has a tight review radius. Examples include converting repetitive assertions to a table, adding another parser branch that follows adjacent cases, spelling out a familiar API call, or completing serialization code from a visible schema. The developer can reject a wrong suggestion in seconds because they already know what right looks like.

Set a practical cutoff. When you must ask whether another file controls the behavior, stop accepting large completions and search. When the edit crosses a persistence boundary, permission boundary, language boundary, or asynchronous job boundary, switch to an assistant for analysis or an agent for investigation. The switch is based on uncertainty and blast radius, not line count. A one-line change to a record layout can be riskier than a hundred-line test fixture.

Review completion output as code from a fast colleague who has not attended the design meeting. Check error paths, resource ownership, numeric conversion, encoding, and concurrency assumptions. Do not reward the tool for matching local style if the local style carries a bug. Repetition is precisely where completion can reproduce a bad pattern efficiently.

Disable or constrain completion where accidental disclosure or insertion is unacceptable. Regulated source, secrets in nearby configuration, generated legal text, and production consoles deserve explicit handling rules. The relevant question is not whether the model provider is generally trustworthy. It is what data the host sends, where inference runs, what it retains, and which controls your environment can enforce.

Assistants are strongest when the question can be bounded

An assistant works well when a developer can define the evidence set and evaluate the answer without granting write access. It is a good fit for code archaeology, design comparison, patch review, query explanation, and converting an incident observation into test cases.

Give it a bounded question with named evidence. Instead of "explain this subsystem," ask: "Using the job definition, these two copybooks, and the three callers, explain when CUSTOMER-STATUS changes from H to A. Cite the file and symbol for each transition, and list any path you cannot resolve." The answer may still be wrong, but the requested shape makes unsupported leaps visible.

The assistant should separate observation, inference, and proposal. Observation says a caller passes a blank field. Inference says the blank probably represents a missing amount because two tests expect that result. Proposal says the new parser should map blanks to an explicit optional value. Mixing those statements turns a plausible design choice into a supposed fact about the old system.

Use conversation for adversarial review. Ask what breaks if records arrive out of order, if the job restarts after a write, if a string contains a non-ASCII character, or if the database call commits independently. Then verify the answers in source or runtime evidence. The assistant is useful because it can enumerate paths humans skip when tired, but a question it invents is not evidence that the path exists.

Avoid endless chats that accumulate stale assumptions. Once the evidence set changes materially, start a fresh analysis with the corrected facts and a concise record of decisions. Otherwise an early misreading can remain in the conversation and influence later answers after the team believes it has been corrected. Store durable findings in tests, architecture notes, or issue records, not solely in chat history.

Agents need bounded authority and visible evidence

An agent earns broader scope by making its actions inspectable and reversible. Give it the minimum permissions needed for the task, an isolated branch or worktree, deterministic setup instructions, and acceptance checks that fail loudly. Keep production credentials and deployment rights outside the loop unless the task explicitly requires them and a human approval gate exists.

A useful run report should answer concrete questions:

• What repository state and instructions did the agent start with?
• Which files did it read and change?
• Which commands ran, and what were their exit results?
• Which acceptance conditions passed or failed?
• What uncertainty remains, and what evidence would resolve it?

The diff remains necessary, but it is not sufficient. Reviewers also need the path that produced it. An agent may delete a test to make a suite pass, update a snapshot that captured a regression, or choose a fallback configuration that never runs in production. Command transcripts and before-and-after test counts expose some of these shortcuts. Repository policy should forbid others.

Contain failure mechanically. Limit writable paths where possible. Require approval before destructive commands, dependency changes, network access, or modifications to deployment configuration. Cap task duration and changed-file count as tripwires, not as definitions of correctness. If the tripwire fires, preserve the work and evidence for review rather than letting the agent widen its own authority.

Security and compliance need precise language. Running a model inside the customer perimeter can address data-location and network constraints, but it does not confer a certification on the tool or the resulting system. An air-gapped setup still needs access controls, audit records, model and dependency provenance, and a process for approving what leaves the environment.

Choose the tool by evidence and blast radius

The choice should follow the claim you need to make. If the claim is "this line matches the pattern I already understand," autocomplete may be enough. If the claim is "these files imply this behavior," use an assistant and verify its cited evidence. If the claim is "the repository now satisfies this acceptance condition," an agent can gather the evidence, provided its tools and permissions cover the condition.

Do not buy the category label. Ask vendors to demonstrate the actual observation and action boundary. Can the tool read every language in the tree? Does it follow generated and configured edges? Can it run the build and parity checks in your environment? Does it show commands and failures? Can you restrict writes, network use, and credentials? What exactly causes it to stop? A polished patch answers none of those questions.

For an inherited system, begin the tool decision with a risk table, not a feature list. Put behavioral ambiguity on one axis and blast radius on the other. Low ambiguity and low impact favor completion. Bounded ambiguity with a human-owned decision favors an assistant. High ambiguity or a cross-system change calls for an agent plus an independent oracle, and sometimes it calls for delaying the change until the team can capture that oracle.

CodeHero uses the last model for legacy rewrites: its platform reads the whole multilingual codebase, modernizes the architecture, and checks behavior against recorded production traffic with a parity harness. It delivers each project in under 30 days, but the schedule does not relax the evidence requirement; the action loop and the oracle are the reason that schedule can be argued about in engineering terms.

Procurement should use a task that contains at least one misleading local pattern, one cross-language dependency, and one behavior visible only at runtime. Give every candidate the same repository state and acceptance evidence. Compare unsupported claims, excluded files, commands, failed attempts, and reviewer effort as carefully as the final patch. A tool that admits an unresolved edge is safer than one that fills the gap with a fluent assumption.

Ownership after the run matters too. The team must be able to reproduce the checks, maintain the replacement, and understand the remaining exceptions without access to a vanished chat session. Require ordinary source, tests, run instructions, and decision records. If a vendor cannot hand over the evidence chain in forms your engineers can inspect, the system remains unfamiliar after the rewrite, only in a newer language.

A tool should never receive more authority than its evidence can justify. When the code is unfamiliar, inspect what the tool saw, what it did, and how it tried to prove the result. The three names matter because they force that conversation before a fluent suggestion becomes an unexamined production change.