Go rewrite infrastructure costs follow the workload

A rewrite to Go can cut an infrastructure bill, but the language name on the deployment manifest does not do the cutting. Savings appear when the new program keeps less live state, creates less temporary garbage, admits work deliberately, reuses connections and lets operators run fewer instances at the same service level. If those mechanisms do not change, the invoice may not change either.

I have seen teams celebrate a binary that used one quarter of the memory in a developer test, then discover that production still needed the old instance count. The constraint was a batch deadline, a database connection cap or an availability rule rather than heap size. A credible business case therefore starts with four separate quantities: memory per instance, elapsed batch time, connections per instance and required instance count. Put a price beside them only after measuring all four.

The bill moves only when a constraint moves

Infrastructure cost falls when a measured resource reduction crosses a purchasing boundary. Dropping resident memory from 1.8 GB to 900 MB matters if it lets a service move from a 4 GB shape to a 2 GB shape, place twice as many processes on a host, or remove instances. It has no direct cash effect if policy still assigns 4 GB, the cluster has spare capacity, or a licensed dependency sets the cost.

Start with the workload's binding constraint. An online service may be limited by latency at peak request rate. A nightly settlement job may be limited by its completion deadline. A desktop-era server moved into containers may be limited by one session's state and the rule that every instance must survive another instance failing. These are different sizing problems even when the source repository is the same.

Separate utilization from allocation as well. Cloud and container invoices usually reflect requested or provisioned capacity, not the lowest number visible in a profiler. A process that uses 600 MB but requests 2 GB still occupies 2 GB in the scheduler's model. After a rewrite, someone must change requests, limits, autoscaling thresholds and host packing rules. Otherwise an engineering win remains stranded capacity.

The useful comparison is cost per completed unit under the same service objective: a request within the same latency percentile, a policy calculated correctly, or a batch finished before the same cutoff. Comparing idle processes proves almost nothing. Comparing unlike reliability targets is worse, because the cheaper system may simply carry less headroom. Build the baseline from billing records and deployment configuration, not recollection. Record instance shapes, replica floors, autoscaling ceilings, memory and CPU requests, node reservations, database tiers and scheduled job duration. Tag each item as variable, stepwise or fixed. A variable charge follows use, a stepwise charge changes only when you cross a tier, and a fixed commitment does not move during its term. This classification stops a percentage improvement in one process metric from being copied carelessly into the budget.

Ownership matters after the measurement. The rewrite team can demonstrate a lower safe request, but a platform owner may control container defaults and finance may control reservations. Put the configuration change, its owner and the earliest purchase date in the same acceptance record as the benchmark. Otherwise everybody agrees that capacity improved while nobody changes the unit being bought.

Lower memory comes from less retained state

Go often reduces memory per instance when the rewrite replaces a heavyweight runtime and object graph with compact data structures and explicit ownership boundaries. The mechanism is not that Go has magically cheap memory. The new service may load less configuration, discard request data sooner, stream records instead of materializing them, and represent domain values without layers of framework objects.

Measure three different things: live heap after garbage collection, total process resident set size, and the container or operating system working set. The live heap tells you how much reachable Go data remains. Resident memory also includes goroutine stacks, runtime metadata, executable mappings, native allocations and pages the operating system has not reclaimed. A procurement model based only on HeapAlloc will be optimistic.

Go's stacks begin small and grow, which helps workloads with many mostly idle concurrent operations. Goroutines are still not free. Each blocked request can retain a stack, buffers and references to a large object graph. A service with no admission limit can turn a traffic spike into thousands of parked goroutines and preserve far more memory than the steady-state profile suggested.

Allocation rate matters alongside the live set. A handler that allocates 20 MB and releases it all may leave little live heap, yet make the garbage collector run frequently and consume CPU. Reusing scratch buffers can help, but a global sync.Pool is not a storage policy. Objects in it may disappear at a garbage collection, and oversized buffers can make the retained footprint erratic. Put size ceilings on anything returned to a pool.

Use GOMEMLIMIT as a runtime guardrail, not as a promise that resident memory will remain below that number. The Go runtime documentation describes it as a soft memory limit for memory managed by the runtime. Native libraries, memory mapped files and other process pages sit outside that accounting. Set the limit below the container limit, leave room for those pages, then observe garbage collection CPU under real load. A limit that is too tight trades an out-of-memory kill for continuous collection and poor latency.

Memory measurements need production traffic

A representative memory result requires the production mix, warm caches and enough run time to expose retention. Synthetic requests that all follow one happy path miss large reports, rare tenant configurations, retry bodies and the slow clients that keep response buffers alive. Replay recorded traffic where policy permits, or build a scrubbed corpus that preserves payload sizes and route frequencies.

Take measurements at defined phases. Record the process after startup, after cache warmup, at sustained normal load, at peak load and after traffic returns to normal. The last point catches retention: if the live heap falls but resident memory stays high, the runtime may be holding reusable pages; if the live heap stays high, your code still reaches the data. Those lead to different fixes.

A small capture script makes review less subjective:

curl -s http://127.0.0.1:6060/debug/pprof/heap > heap.pb.gz
go tool pprof -top -sample_index=inuse_space ./service heap.pb.gz
cat /sys/fs/cgroup/memory.current

The profile output lists functions with flat and cumulative in-use bytes. The cgroup file prints one integer in bytes. Capture the deployment's requested memory beside both results, because that is the number a scheduler and often the bill care about. In environments without the cgroup v2 path, use the platform's container working-set metric and document its definition.

Do not expose net/http/pprof on a public listener. Bind diagnostics to a private administrative endpoint or collect a profile through the platform's authenticated mechanism. A heap profile can reveal type names, allocation patterns and fragments of process state, so treat it as operational data rather than a harmless chart.

Compare profiles by route and workload phase, not just by top allocator. One large startup allocation may dominate a cumulative view but have no relation to peak growth. Labels attached through pprof.Do can separate batch stages or request classes and show which work holds memory when the process approaches its limit.

Retained memory deserves an allocation graph, not a reflexive cache purge. Use the profile's paths to roots to find the map, channel, timer or goroutine that keeps data reachable. A missing cancellation call can preserve an entire request context. A metrics label built from customer input can create an unbounded series map. A cache may have an entry limit but no byte limit, so a few unusually large values break the assumption behind sizing. Fix the ownership rule, then repeat the same traffic phase.

Run long enough to cross periodic work. Certificate reloads, report generation, compaction and daily reference-data refreshes can establish a higher plateau than an hour of request replay. If the service has a weekly or month-end path, capture it separately and size for the policy around that event. Averaging it into ordinary traffic hides the exact peak that causes a kill or an emergency scale-out.

A faster batch needs bounded parallel work

Go shortens a batch window when the old program leaves CPU or I/O idle and the new program overlaps independent work without overwhelming the next system. Replacing a sequential COBOL, PL/SQL or desktop loop with goroutines can expose parallelism, but unrestricted fan-out often makes the run slower.

The correct concurrency limit comes from the narrowest shared resource: database sessions, storage throughput, a remote service's quota, CPU, or lock contention. Use a fixed worker count or a semaphore, record queue time separately from execution time, and cancel the group when the job can no longer succeed. That gives operators a knob they can reason about.

For example, a reconciliation job may read account partitions independently but write through one indexed ledger table. Raising workers from four to sixteen can shorten the read phase. Raising them to two hundred may saturate the database, increase lock waits and turn every operation into a longer operation. The program then holds more rows, buffers and goroutine stacks at once, so the attempted speedup raises both elapsed time and memory.

Batch timing also includes startup, checkpointing, retries and finalization. A benchmark that measures only the inner loop can claim a large improvement while the cutoff remains unchanged. Report the wall-clock interval from the scheduler releasing the job until downstream consumers can safely use its output. Keep the same input volume and correctness checks.

Preserve restart behavior during the rewrite. If the old job commits every partition and the new one commits only at the end, a clean run may look faster while one late failure forces a complete rerun. The expected infrastructure cost then depends on failure frequency and repeated work. Idempotent writes, durable checkpoints and bounded retry budgets are performance controls because they decide how much work the system repeats.

Some batches will not improve much. A job already saturating one storage channel cannot outrun that channel merely because its control logic is now Go. A licensed mainframe feed may release input at a fixed time. A downstream system may accept one file at a time. In these cases a rewrite can improve maintainability and recovery without shrinking the scheduled window, and the financial model should say so.

CPU efficiency needs its own check. A Go worker that decodes text, converts decimal values and builds short-lived structs may spend more CPU than expected even while it finishes sooner through concurrency. Measure total CPU-seconds for the complete batch as well as elapsed time. Elapsed time tells the operations team whether the cutoff is safe; CPU-seconds tell the capacity model how much shared compute the run consumes. One can improve while the other worsens.

Control memory with backpressure between stages. If readers can outrun writers, an unbounded channel quietly becomes an in-memory copy of the input. Give every queue a capacity derived from item size and acceptable buffering, and make producers block or spill to a durable store when it fills. Then expose queue depth and blocked time. That design makes the batch's peak memory predictable and shows whether another worker would help or merely move waiting upstream.

Connection handling can erase the gain

Connection pools convert instance count into pressure on databases and remote services, so a smaller process can still cost more if each copy opens too many connections. Go's database/sql pool and net/http.Transport reuse connections, but their defaults are not a capacity plan. Operators must set limits that match total fleet size and upstream behavior.

For SQL, treat SetMaxOpenConns as a fleet budget. If the database permits 400 application sessions and the service can scale to 20 instances, a nominal maximum of 20 per instance consumes the entire allowance before migrations, administrative access or failover headroom. The arithmetic must use maximum possible instances, not today's average. SetMaxIdleConns controls how many ready sessions each process holds, while SetConnMaxLifetime and SetConnMaxIdleTime retire sessions over time.

A low open-connection limit creates a queue inside the process. That may be correct, but observe DB.Stats(): WaitCount and WaitDuration show whether callers waited for a slot. If latency grows there, adding application instances may make the database queue worse because every new process adds another pool. Fix the query, transaction length or database capacity before autoscaling the symptom.

HTTP has a separate trap. Creating a new http.Client or Transport for every request defeats reuse and causes connection churn. Reuse a configured transport, close response bodies on every path, and read or drain bodies when reuse requires it. Set idle limits per host, response header timeouts and an overall request deadline that matches the operation. A large global idle pool multiplied by many instances can keep thousands of sockets open even at modest request volume.

Connection churn has costs outside the process. TLS handshakes consume CPU, short-lived sockets accumulate in operating system tables, and a database may spend work authenticating sessions that vanish immediately. These effects explain a common puzzle: the Go service uses less heap, yet database CPU rises and the instance count cannot fall because tail latency is worse.

Transactions make pool arithmetic more subtle. A handler that opens a transaction, calls a remote service and then commits owns a database session during the network wait. Twenty open connections may support hundreds of quick queries or only twenty stalled transactions. Keep remote calls outside transactions where correctness allows, set statement and transaction deadlines, and log duration without logging sensitive arguments. Pool size cannot repair a transaction boundary that is too broad.

Failover tests should watch connection storms. When a database endpoint changes or a remote server closes idle sockets, every application instance may reconnect together. Tight retry loops multiply authentication and discovery work just as the dependency recovers. Use capped exponential backoff with jitter, respect the caller's deadline and limit concurrent dials. Measure recovery time and upstream load at the fleet's maximum instance count, because a single-process test cannot reveal the synchronized surge.

Instance count follows throughput and failure policy

The required instance count is the largest count demanded by throughput, memory, latency and availability, rounded up with explicit headroom. Do not divide old memory by new memory and call the quotient a consolidation ratio. That calculation ignores whether one instance can process the offered load and whether the fleet can lose a member.

Build a small capacity record for each critical service. It needs peak arrival rate in requests per second, safe throughput per instance before latency breaks, working set at that throughput, maximum connections per instance and the number of failed instances policy requires the fleet to tolerate. Keep the raw measurements beside the chosen values so reviewers can see where a safety margin entered the calculation.

If peak arrival is 1,200 requests per second and one instance safely sustains 275 at the required latency, throughput needs ceil(1200/275) = 5 instances. If policy requires the service to survive one failed instance while serving that peak, deploy at least six. These numbers are illustrative arithmetic, not a promise about Go performance. Measure your value at the point just before latency or errors leave the service objective.

Autoscaling does not remove this work. A CPU target can scale a CPU-bound handler reasonably, but it may miss a service waiting on a saturated connection pool. A memory target can also react too late because retained memory falls slowly. Choose a signal tied to the constraint, such as queue delay or concurrent work, and include startup time in the reserve. A tiny binary that needs several minutes to warm a large cache still needs spare instances before the spike arrives.

Host packing creates another boundary. Smaller memory requests save money on a fixed cluster only when the scheduler can place enough workloads to remove a node or defer the next node. Fragmented CPU and memory requests can leave unusable gaps. Repack the whole node pool on paper, then test scheduler behavior, before booking the saving.

Use separate counts for normal operation, failover and deployment. A rolling deployment may temporarily run old and new replicas together. A regional evacuation may put traffic on a fleet that normally serves half as much. If the cost proposal sizes only the quiet steady state, the first deployment or failover will either violate the objective or force operators to restore the old limits. Temporary capacity can still be cheaper, but it must exist in quotas and the cost forecast.

Beware averages in multitenant work. Two instances can show the same average CPU while one holds a customer with a huge working set and the other handles many small customers. Before lowering memory, replay the heaviest allowed tenant mix or add placement rules that stop several large tenants landing together. If the product contract has no tenant size bound, the system needs admission rules or a defensible worst-case allocation.

Some rewrites cannot lower the invoice

Savings do not appear when infrastructure is a small or fixed part of the system's cost, or when the new design inherits the old constraint unchanged. A rewrite can still be the right decision, but attaching an unsupported compute saving weakens the proposal.

The clearest non-saving cases are these:

• Minimum availability already sets the fleet at two or three instances, and load would fit on one in either language.
• A database, message broker, vendor license or reserved host dominates the bill.
• The workload spends nearly all its time waiting on an unchanged serial dependency.
• Data residency or isolation rules require a dedicated environment for each customer regardless of utilization.
• Traffic is so intermittent that serverless billing, startup time or minimum platform charges dominate runtime efficiency.

Cgo and native libraries can narrow the memory difference because their allocations may sit outside Go's heap controls. A rewrite that wraps the same native calculation engine may change orchestration without changing the expensive work. Likewise, moving a monolith into many services can duplicate caches, transports, telemetry buffers and minimum replicas. The total fleet may use more memory even when every individual process looks lean.

Observability can be material too. High-cardinality labels, unbounded trace queues or verbose payload logging consume memory, network and storage in any language. If the rewrite adds modern telemetry that the legacy program lacked, compare like with like or list the added capability as a conscious new cost. Do not hide it inside a language benchmark.

A final source of disappointment is transliteration. Reproducing every old process boundary and in-memory table in Go preserves the architecture that created the footprint. The code may compile to a smaller executable, but the system still loads the same data, waits on the same locks and starts the same number of copies. The mechanism must change somewhere for the bill to move.

Unit prices can move against the technical result. A managed platform may charge more for fewer, larger instances than for the existing reserved shapes. Network charges can rise when a split service crosses availability zones for calls that used to stay inside one process. A new Postgres deployment can replace a sunk database license with a visible managed-service line. Price the target topology itself rather than multiplying current unit prices by a hoped-for instance ratio.

Staff time is separate from infrastructure even when both belong in the rewrite decision. Simpler deployment, faster incident diagnosis and removal of scarce-language dependencies may dominate the return. Keep those claims in their own model with evidence that fits them. Blending them into an inflated compute number makes a sound modernization look less credible when the first cloud invoice arrives.

A parity test belongs beside the load test

Performance results count only after the new system produces the same business behavior. The cheapest program in a benchmark is useless if it rounds money differently, changes sort stability, loses a retry or treats a blank field differently. Run correctness and capacity tests against the same corpus so nobody can trade one for the other quietly.

Record inputs and externally visible outputs from the current system, remove or protect sensitive data, then replay the corpus through both versions. Compare response status, relevant headers, normalized bodies, database effects and emitted events. Normalize values that are meant to differ, such as generated identifiers or timestamps, with named rules under review. Every unexplained difference blocks the performance claim.

For online traffic, ramp load until the service objective fails, not until the process crashes. Capture throughput, latency distribution, errors, working set, allocation rate, garbage collection CPU, pool waits and upstream load at each plateau. For batch work, replay several representative input sizes and record stage timings, checkpoints, retry work and the final availability time.

Keep the experiment configuration with the result. At minimum it should identify the build, runtime settings, CPU and memory limits, worker counts, pool limits, input corpus and upstream versions. A result without those values cannot be repeated and will decay into folklore during budget review.

Run one deliberate saturation test after the normal plateaus. The aim is not a heroic maximum. It is to observe whether overload stays bounded: queues should stop growing, callers should receive controlled errors or backpressure, memory should settle, and recovery should occur without a restart. Record the first constraint reached. That constraint belongs in the capacity sheet and should drive the autoscaling signal.

Repeat the chosen safe plateau on more than one fresh instance. Runtime warmup, host contention and corpus order can change a single run. Report the spread and retain the raw time series. You do not need a decorative benchmark score; you need enough repeated evidence to choose a request and replica count that operators will trust at 2am.

CodeHero uses a parity harness against recorded production traffic when rewriting a legacy system, then modernizes the architecture instead of transliterating it. That is the right order for cost work: lock down behavior, change the mechanisms, and measure the operational boundary that produces the invoice.

Book savings only after changing allocation

A verified reduction becomes financial only when the deployment configuration and purchasing model reflect it. Lower memory requests, adjust CPU requests, reset autoscaling limits, reduce pool budgets where fleet math allows, and test failure behavior at the new minimum count. Then watch at least one full business cycle, including the heaviest batch and peak online interval available in that cycle.

Keep a before-and-after ledger with provisioned capacity and unit prices rather than profiler percentages. Include database tiers, network transfer, observability storage and reserved capacity that remains committed. If a three-year reservation cannot be reduced, call the near-term result released capacity and state when cash can follow. Finance teams understand that distinction; pretending stranded capacity is immediate savings creates trouble later.

The strongest rewrite proposal gives a range. The conservative case applies measured per-instance improvements but preserves current minimum replicas and fixed services. The expected case changes instance shapes or node count after the load and failure tests pass. The upper case belongs in the appendix until real traffic sustains it. Each case should name the constraint that must move.

Go gives engineers good tools for compact services, controlled concurrency and reusable connections. It does not repeal queueing, upstream limits or availability policy. If the capacity sheet cannot connect a lower heap, a shorter batch or a smaller pool to fewer purchased units, keep working on the design and do not book the saving yet.