Typed tools keep agents out of your API's blind spots

An agent should choose what to do. It should never invent how your API wants it done. That division sounds obvious until a model sends customer_id where the endpoint expects accountId, turns a preview into an update, or fills an unknown enum with a plausible word. The request may read well in a transcript and still be invalid, ambiguous, or dangerous.

Typed tools move that ambiguity out of the prompt and into an enforceable contract. The model receives a bounded set of operations, each with a machine-checkable input shape. Your application validates the call before it touches business logic, then asks a human to approve any operation that changes state. The model still reasons about intent. Code owns syntax, authority, and execution.

I have seen teams treat a detailed system prompt as if it were an interface definition. It is not. Prose can explain policy, but it cannot reject an extra field, enforce a discriminated union, compare a version number, or stop a retry from charging twice. If an agent can reach a production API, those controls belong in code.

A prompt describes intent, a tool contract defines permission

A prompt can tell an agent to update a customer only after confirmation. A tool contract defines exactly which update exists, which fields it accepts, and what confirmation means. Those jobs overlap in conversation, but they have different failure modes. Prose fails through interpretation. Contracts fail visibly through validation, which is the kind of failure you can test and operate.

Suppose an internal API exposes one broad endpoint called execute_action. Its arguments are action, resource, and payload, all strings. The prompt lists allowed actions and includes examples. That design feels flexible because a new action needs no schema change. It is also a tunnel around every constraint the API already learned to enforce. The model can misspell an action, send serialized JSON inside payload, or combine a resource with an action that was never meant for it.

A typed surface should expose narrow operations such as get_customer, preview_address_change, and commit_address_change. Each name carries one capability. Each input schema limits the model to fields that operation can use. If the model needs an unsupported action, the call should fail as unsupported. A rejected call is safer than a guessed one, and it tells you where the tool catalog needs work.

This is also where teams confuse type safety with prompt formatting. Asking the model to reply with JSON improves parsing. It does not make the JSON valid for your business. Syntax says the braces match. A tool contract says country uses an allowed code, customer_id identifies the right kind of record, and a write requires an approved proposal. You need both layers.

Keep descriptions, but give them a smaller job. A description explains when to use a tool and what its terms mean. The schema decides what can cross the boundary. When a constraint matters after the model stops generating text, encode it where the executor can check it.

Good schemas make illegal states hard to express

A useful schema does more than label fields as strings. It encodes the choices that change behavior and refuses combinations that make no sense. If an API accepts either an existing shipping address or a new address, model that as two distinct cases. Do not accept twelve optional fields and hope the prompt explains which six belong together.

This JSON Schema fragment gives the model one explicit choice and closes the object against invented fields:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["customer_id", "destination"],
  "properties": {
    "customer_id": {"type": "string", "minLength": 1},
    "destination": {
      "oneOf": [
        {
          "type": "object",
          "additionalProperties": false,
          "required": ["kind", "address_id"],
          "properties": {
            "kind": {"const": "saved"},
            "address_id": {"type": "string"}
          }
        },
        {
          "type": "object",
          "additionalProperties": false,
          "required": ["kind", "line1", "city", "country"],
          "properties": {
            "kind": {"const": "new"},
            "line1": {"type": "string"},
            "city": {"type": "string"},
            "country": {"type": "string", "pattern": "^[A-Z]{2}$"}
          }
        }
      ]
    }
  }
}

The kind field is a discriminator. It prevents a saved address identifier from drifting into the new-address case and gives validation errors a useful location. additionalProperties: false matters because models often produce helpful-looking extras. Silently ignoring those fields trains everyone to accept a mismatch between the transcript and the action that actually ran. Reject them.

Do not encode facts that require live data as static enums. A list of warehouse IDs, user IDs, or current plan names goes stale. Put stable vocabulary such as draft, approved, and cancelled in the schema. Resolve changing identifiers through a read tool, then validate them against the system of record at execution time.

Dates, money, and quantities deserve explicit representations. Use an ISO date string if the API means a calendar date, not a timestamp with an implied timezone. Represent money as an integer in the smallest supported unit plus a currency code, unless the existing domain model dictates another exact representation. Add minimums, maximums, string lengths, and patterns when the domain has them. Every omitted bound becomes a value the agent may reasonably try.

Schema versioning should be boring. Give each tool a version in the registry, keep old versions available while active runs can still call them, and make breaking changes under a new version. Changing a field from optional to required in place can turn a routine agent retry into a mysterious validation failure.

Validate before and after business logic

Validation at the boundary needs two passes. First validate the model's arguments against the published tool schema. Then validate domain facts inside the service that owns them. The first pass catches malformed calls. The second catches calls that are well formed but no longer true.

A request for customer_id: "C-1842" can satisfy every JSON rule while referring to a deleted record or a customer outside the operator's tenant. A positive quantity can exceed available stock. An approved proposal can have expired. The tool adapter must not treat schema success as authorization or domain validity.

Return errors as typed results, not paragraphs that the model must reinterpret. A stable error envelope gives the planner enough information to recover without exposing stack traces:

{
  "ok": false,
  "error": {
    "code": "VERSION_CONFLICT",
    "message": "Customer changed after the proposal was created",
    "retryable": false,
    "field": "expected_version"
  }
}

The code is for control flow. The message is for the transcript and operator. The retry flag tells the runtime whether repeating the identical call could ever help. Keep these meanings stable across tools. If every adapter invents its own error prose, the model becomes your accidental error parser.

Validate outputs too. Tool authors change code, upstream APIs return partial data, and serializers leak fields. An output schema can stop a tool from feeding credentials, internal notes, or an unexpected megabyte of text back into the model context. It also catches the unpleasant case where execution succeeded but the result shape changed and the agent now reasons from missing fields.

Log the validation outcome with the tool name, schema version, run ID, and error code. Do not log raw arguments by default. Tool inputs often contain the exact personal or operational data you are trying to control. Record hashes or selected safe fields when they provide enough evidence.

Reads and writes belong in different capability sets

Classify tools by effect before the model sees them. A read returns information without changing durable state. A write creates, updates, deletes, sends, publishes, pays, deploys, or triggers another system that does one of those things. The HTTP verb is not a trustworthy classifier. A GET endpoint can mark a message read, and a POST endpoint can perform a pure search. Classify the business effect.

Give exploratory agents read tools by default. Add write tools only to the run that needs them, under an identity with matching server-side permissions. Hiding write tools in the prompt is not permission control. If the runtime can still dispatch a named call, prompt injection or a planning error can find it. The dispatcher should reject any tool absent from the run's capability set.

Writes also need narrower shapes. A generic update_record tool asks the agent to understand every table and mutable column. Expose business operations such as suspend_invoice_delivery or change_shipping_address. The service can then enforce invariants, produce a useful preview, and attach an approval policy to that exact effect.

Some operations look reversible but are not. Sending an email cannot be recalled reliably. Publishing an event may start several downstream jobs. Deleting a newly created record may not undo the notification already sent about it. Treat external communication and downstream triggers as writes even when your local database stays unchanged.

For a mixed workflow, split planning from execution. The agent can read records, calculate a proposed change, and ask a preview tool to price or validate it. The final commit tool accepts a proposal identifier, not a fresh free-form payload. That one design choice stops the approved operation from changing between the screen and the write.

Approval must bind to an exact proposed write

An approval button alone provides weak control. The approval record must say who approved what, against which version of the target, and until when. Otherwise a model can receive approval for one payload and execute another, or execute the approved payload after the underlying record has changed.

Use a proposal object created by trusted code. The agent supplies candidate arguments to a preview tool. The service validates them, resolves defaults, calculates consequences, and returns a canonical proposal. The user sees the canonical effect, not the model's conversational summary. A practical approval record can look like this:

{
  "proposal_id": "p_7f31",
  "tool": "commit_address_change.v2",
  "arguments_sha256": "8be7...a91c",
  "target": {"type": "customer", "id": "C-1842", "version": 17},
  "effect": "Replace the shipping address for customer C-1842",
  "expires_at": "2026-08-14T16:30:00Z",
  "approved_by": "user_291"
}

The commit endpoint loads this record, verifies the approver's authority, checks expiry, compares the target version, and hashes the canonical arguments again. It should not accept replacement arguments from the agent. If anything differs, execution stops and the system creates a new proposal.

Approval policy should follow consequence, not tool count. A low-risk draft saved to an isolated workspace may need no human decision. Sending the draft to a customer does. A bulk change, payment, deletion, credential rotation, production deployment, or external message should receive an approval level that matches its reach. Keep the rule in a policy table the runtime can evaluate. Do not bury it across prompts.

The approval screen should show concrete differences: fields before and after, recipients, amount and currency, environment, affected record count, and any irreversible consequence. Do not ask someone to approve run tool call. Approval fatigue starts when the screen hides the effect and forces the operator to trust the agent's summary.

Approvals should expire, and most should be single use. Record denial too, including a short reason the agent can use for replanning. Never turn silence, a closed browser tab, or a timeout into consent.

A write failure can look like success for several minutes

Consider an agent changing a shipping address. It reads customer version 17, proposes a new address, and receives approval. The commit request reaches the service, which writes the address and commits the transaction. Before the response reaches the agent, the connection drops. The runtime sees a timeout. It does not know whether the write happened.

A naive retry sends the same logical change again. If the endpoint appends addresses or emits a fulfillment event, the second request can duplicate work. If the runtime instead reports failure, the operator may repeat the change manually. The transcript says the tool failed even though production changed. This ambiguous outcome is a normal distributed-systems problem, not a model quirk.

Every write call needs an idempotency key generated outside the model. Bind it to the run, proposal, and operation. The service stores the key with the final result in the same transactional boundary as the write when possible. A retry with the same key returns the stored result. A call that reuses the key with different arguments must fail.

The runtime should handle the timeout in a fixed sequence:

1. Query the operation status by idempotency key.
2. If the service recorded success, return that typed result to the agent.
3. If the service recorded a terminal failure, return the recorded error.
4. If status is unknown, pause and escalate rather than inventing an outcome.

Optimistic concurrency closes another hole. The proposal above targets version 17. If a person changes the address before commit, the current version becomes 18 and the commit fails with VERSION_CONFLICT. The agent must read the new state and create a fresh proposal. Reusing the old approval would apply a decision made against facts that no longer exist.

Automatic retries are appropriate for reads that declare themselves safe and for writes protected by idempotency with a known status protocol. Do not let a generic retry library decide this from network errors alone. The tool definition should publish its retry class, and the executor should enforce it.

Tool results should contain evidence, not a victory sentence

A successful tool response needs enough structured evidence for the next decision. Done is not enough. Return the resource identifier, its new version, the operation ID, the fields that changed, and any next state the workflow depends on. Keep display prose separate from control fields.

For the address change, a useful result might be:

{
  "ok": true,
  "operation_id": "op_a812",
  "customer_id": "C-1842",
  "previous_version": 17,
  "new_version": 18,
  "changed_fields": ["shipping_address"],
  "committed_at": "2026-08-14T16:22:11Z"
}

That response lets the agent report what happened without making it up. It also lets a later step pass new_version into another proposal. If the service returns a human message, treat it as display text, never as the sole evidence of success.

Limit result size deliberately. A search tool should return a bounded page and a cursor, not every matching row. A file tool should return metadata and a handle when the content exceeds the model's working need. Large untyped results increase cost and make prompt injection inside retrieved data harder to isolate. Mark tool data as untrusted content in the runtime, even when it came from your own database; stored text may have originated with an attacker.

Redact at the adapter, before the result enters the model context. Permission to call get_customer does not imply permission to reveal every customer column. Define a result view for the task and keep secrets, internal flags, and unrelated personal data out of the schema. Output validation then guards that view against regressions.

For long operations, return an operation resource with a finite state enum such as queued, running, succeeded, failed, or cancelled. Poll through a read tool. Do not keep a model call open while a deployment or migration runs, and do not let the agent infer success from elapsed time.

Retries, cancellation, and concurrency need declared semantics

A tool registry should describe operational behavior alongside input and output schemas. At minimum, record whether the tool reads or writes, whether identical calls are safe to retry, whether it supports idempotency, what approval policy applies, and how cancellation works. Those are executor rules, not prose hints for the model.

Cancellation deserves precision. Cancelling an agent run can stop future tool calls, but it cannot automatically undo a request already accepted by another service. A cancel endpoint should return whether the operation was stopped, had already completed, or cannot be interrupted. If compensation exists, expose it as a separate write with its own preview and approval. Do not label compensation as rollback when it creates another business event.

Concurrency limits belong at several levels. Cap calls per run so a planning loop cannot flood an API. Cap calls per tenant so one busy workflow cannot starve others. Add resource-level serialization when two approved writes to the same record would conflict. The existing service remains responsible for transactions and locking; the agent runtime does not replace database correctness.

Timeouts should express the tool's behavior. A two-second lookup and a long numeric conversion should not share one arbitrary deadline. CodeHero's agentic platform reads whole legacy codebases in parallel, while parity is checked against recorded production traffic; that kind of workload needs bounded operations and explicit completion state rather than conversational guesses.

Rate-limit errors should say when another attempt may succeed, but the runtime must still respect the run's deadline and approval validity. If an approved proposal expires during backoff, the next call should fail and request fresh approval. Convenience does not outrank the consent boundary.

Contract tests catch failures prompt tests miss

Prompt evaluation can tell you whether the model usually selects the right tool. Contract tests prove the wrong call cannot execute. You need both, but the second set protects production when the model, prompt, or tool description changes.

Build fixtures from real boundary cases. For each tool, test the smallest valid request, unknown fields, missing required fields, wrong union branches, bounds, stale versions, expired approval, an approver without authority, duplicate idempotency keys, and a valid key reused with different arguments. Check output validation and redaction with the same discipline.

A compact contract test can read like this:

GIVEN proposal p_7f31 targets customer C-1842 version 17
AND the current customer version is 18
WHEN commit_address_change.v2 executes with idempotency key run9:p_7f31
THEN no address is changed
AND the result code is VERSION_CONFLICT
AND the proposal remains unconsumed

The last assertion matters. If a conflict consumes the approval, the workflow needs a new approval after replanning, which may be correct. If your policy allows the same approval to survive a transient service failure, define that separately. Tests force the team to settle the distinction instead of discovering it during an incident.

Test the dispatcher as an adversarial boundary. Ask for an unregistered tool, a write tool in a read-only run, an old schema version, an oversized argument object, and strings containing instructions aimed at the runtime. The dispatcher should parse data, enforce limits, and call only a registered handler. It should never evaluate model-generated code or construct a method name dynamically.

Keep a small set of end-to-end traces too. Record the tool catalog, model request, proposed calls, validation decisions, approvals, service results, and final response with sensitive values removed. Replay those traces after schema changes. Exact wording may vary, but the permitted effects and invariants should stay fixed.

Contract tests should also pin the catalog itself. Save an expected list of tool names, versions, effect classes, and approval policies for each runtime role. A newly registered write then fails review if someone forgets to add a policy, and a supposedly read-only role fails if its catalog gains a commit operation. This catches permission drift before an evaluation prompt happens to select the new tool.

Generate invalid cases systematically, but keep the generator inside schema bounds you understand. For a required string, try omission, empty text, an oversized value, and the wrong primitive type. For a union, combine fields from both branches and supply an unknown discriminator. For numbers, test the exact limits and the nearest value outside each limit. The point is to prove that every declared boundary has an executable rejection path.

Production telemetry should answer concrete questions without storing sensitive payloads. Count calls by tool and schema version, validation failures by code and field, approval decisions, conflicts, ambiguous outcomes, retries by declared class, and output validation failures. A sudden rise in unknown-field errors usually means a prompt or client moved ahead of the registry. Repeated version conflicts may mean proposals live too long or a workflow reads too early. Those signals tell you whether to change a schema, a tool description, or the surrounding sequence.

Treat validation errors as product feedback, not as text to patch over automatically. If the model repeatedly supplies email to a tool that only accepts customer_id, decide whether lookup belongs in a separate read tool or whether the write should accept a stable alternate identifier. Do not quietly add optional fields until calls pass. Every new field expands the operation and needs its own authorization, redaction, and test decisions.

Run fault injection around the executor. Drop the connection after the service commits, return a malformed success body, delay an approval until it expires, race two proposals against the same version, and make the status endpoint temporarily unavailable. Verify that the runtime reports an unknown outcome when evidence is missing. A fabricated success response can look polished in an evaluation, so assert against recorded service state rather than the final sentence alone.

Finally, test that approval display and commit share the same canonical proposal. Render the approval from stored canonical data, approve it, then mutate every agent-controlled copy of the arguments before commit. The committed effect must remain identical to the displayed effect. If that test is hard to write, the approval boundary probably depends on conversational state, which is exactly where it should not live.

The safe default is a smaller tool surface

Start with the narrowest catalog that completes one real workflow. A tool earns its place when its input can be bounded, its output can be validated, its effect can be classified, and its failures can be represented without asking the model to guess. If you cannot define those pieces, the API is not ready to become an agent tool.

Resist the popular advice to expose every internal endpoint and let the model plan freely. Teams like it because the first demo appears quickly. In production it transfers API archaeology, permission selection, and error interpretation into a probabilistic component. The model then spends tokens rediscovering rules your services already know, and one plausible mistake can cross a write boundary.

A narrow tool catalog does not make the agent less capable. It makes capability explicit. Add a tool when logs show a missing operation, not when a prompt grows another paragraph explaining how to squeeze an unrelated action through a generic endpoint. Version the contract, attach policy, and give the executor a typed result.

The standard for a write is higher. Require a canonical proposal, an approval bound to its hash and target version, an idempotency protocol, and a result that proves what changed. Make unknown outcomes visible to an operator. A paused workflow is inconvenient; an agent that confidently reports the wrong production state is expensive.

Typed tools are the point where an agent stops being a chat interface wrapped around privileged credentials and becomes a controllable software component. Keep reasoning in the model. Keep permission and truth at the boundary.