| Takeaway | Detail |
|---|---|
| Tier agent state above roughly 10,000 writes per hour to cut total storage cost about 60%. | Compare tiered hot KV plus cold object storage against keeping every write on a single premium tier; the ~60% saving applies only when the write path is batched. |
| Instrument write rate and read-recency per agent-state key for one week before splitting tiers. | Split into a hot KV tier plus a cold tier only if sustained writes exceed 10,000/hour and more than 80% of keys are unread after 24 hours. |
| HDD workload is measured in terabytes per year; failure rate is how often drives stop working within a year. | Both figures come from HDD workload and failure-rate definitions, not from agent-state benchmarks. |
| Hyperscaler exits can deliver 50–80% savings at real-world cost thresholds and workload signals. | Cost inflection points and migration patterns indicate when moving from public cloud VMs to a hosted private or alternative cloud pays off. |
This guide sets a concrete threshold for tiering agent state: about 10,000 writes per hour, with hot state on a low-latency KV store and cold checkpoints demoted to object storage.
It gives the one-week instrumentation rule — write rate plus read-recency per key — that tells you whether the roughly 60% storage-cost cut applies to your workload.
How agent state tiering actually works
Agent state is the per-session working memory of an agent: conversation turns, tool-call results, and scratchpad variables. It gets written on every step and read on the very next step. Checkpoints are different — periodic full snapshots taken for resume and audit, written rarely and read even more rarely. Treating these two workloads as one storage problem is what makes single-tier designs expensive at volume.
The hot/cold split works like this. The hot tier holds the last N steps for each live agent in a low-latency KV store, sized so that step-to-step reads never touch anything slower. The cold tier holds checkpoints and demoted keys in object storage, addressed through a manifest that maps agent_id plus a step_range to an object key. That manifest is the linchpin: any read of old state goes manifest-first, resolves to an object key, then fetches — the application never has to know which tier answered.
Demotion is a background job, not a request-path operation. It scans each key's age and last-read timestamp; when a key passes your recency threshold without a read, it is serialized into the next object write for that agent and deleted from the KV store, with the manifest updated in the same transaction. Promotion is the reverse: if a key in the cold tier is read, the job fetches the object, rehydrates the key into the hot tier, and serves subsequent reads from there. Both jobs should be idempotent — a crashed demotion mid-write must leave the manifest pointing at either the old KV key or the complete object, never a partial.
The batching condition matters as much as the split. Because demotion appends keys into grouped object writes, unbatched write paths will thrash the cold tier with tiny objects, and the savings described elsewhere in this piece evaporate. Before switching anything on, instrument write rate and read-recency per agent-state key for one week. If sustained writes exceed roughly 10,000 per hour and more than 80% of keys go unread after 24 hours, the split earns its complexity; below those signals, a single tier is simpler and defensible.
Two operational checks keep the mechanism honest. First, alert on manifest-lookup misses — a rising miss rate means demotion and the manifest are drifting out of sync. Second, verify that checkpoint writes still land even when an agent is idle; resume and audit depend on the cold tier being complete, not merely cheap.
The evidence: what the numbers say
Proposals to split agent state arrive with three citations in hand, and it is worth knowing precisely what each one measures. The IBM Z Performance and Capacity Analytics documentation (June 2026 edition) defines its Dynamic Threshold Workload Delay report as statistics on workload delay against a dynamic threshold, by hour, for a particular MVS system ID, policy, service class, and service class period. Read that scope carefully: it is an overload-detection method, bound to a named resource and a named period. It contains no storage cost figure of any kind. If someone cites it as evidence that a premium tier is expensive, they are citing a reporting definition for a claim the document does not make.
Everhour's workload-tracker analysis (June 2025) frames workload as tasks per unit time per resource, with thresholds set explicitly so an overloaded resource is visible rather than inferred from a bad week. Taken as a definition, that is the useful contribution: a sustained rate means a measured rate over a declared window, not the peak observed during a migration. The analysis carries no storage cost data, so it can establish when your fleet is writing heavily and cannot establish what either tier charges for those writes.
Device-level reporting points the same direction. HDD workload is measured in terabytes read and written per year, and failure rate as how often drives stop working within a year (laptopjudge.com, updated September 29, 2026). That is volume accounting at the drive — good for sizing and durability planning, silent on gigabyte-month pricing at the tier above it.
Line the three up and the convergence is narrow but genuine: each source attaches a metric to a named resource and a named period, and none of them carries that metric across into money. The table below records what each actually supports.
| Source (date) | What it measures | Unit | Supports a storage-cost claim? |
|---|---|---|---|
| IBM Z Performance and Capacity Analytics, Dynamic Threshold Workload Delay report (June 2026) | Workload delay against a dynamic threshold, by hour, per MVS system ID, policy, service class, and period | Delay versus threshold | No — overload detection only |
| Everhour workload-tracker analysis (June 2025) | Workload as tasks per unit time per resource, with explicit thresholds | Tasks/time per resource | No — defines sustained rate only |
| HDD workload and failure-rate reporting, laptopjudge.com (updated September 29, 2026) | Data read and written per drive; drives failing per year | Terabytes per year; failures per year | No — device volume, not tier pricing |
So the review question for any tiering business case is a sourcing question, asked twice. Which instrument produced the write rate, and which of your own billing lines produced the cost delta? If the first answer is a projection from a pilot and the second is a published tier comparison, the two halves of the argument came from different measurements — and the headline saving is an assumption wearing a citation, not evidence you can act on.
Options compared: single tier vs split
Three storage strategies compete for agent state, and they differ mainly in where cold checkpoints land and how reads behave. The right choice is not simply the cheapest per gigabyte; it is the one that keeps next-step reads fast while avoiding hot-tier charges for data that is rarely touched.
Option A keeps every write on a single premium KV tier. It has the simplest write path, no demotion job, and no manifest to maintain, but every cold checkpoint continues to accrue hot-tier GB-month charges. That simplicity is attractive when write volume is modest or most keys are read again soon.
Option B splits state into a hot KV tier for active agent state and cold object storage for batched checkpoints. It adds a demotion job and a manifest, but it moves cold bytes to the cheaper object tier, so cold GB-month cost falls by the object-versus-KV price ratio. For qualifying high-write, low-recency workloads, this is the winner: it preserves low-latency reads for live agents while stopping cold checkpoints from paying hot-tier rates.
Option C stores everything in object storage. It is the cheapest per gigabyte, but every step read becomes a full object GET, which breaks the next-step latency budget for live agents. It is better treated as an archive target than as the primary state store.
Compare the options on three axes: write-path complexity, read latency for the next step, and cost of cold bytes. Option A minimizes complexity, Option C minimizes per-GB cost, and Option B is the only one that separates hot reads from cold storage.
| Option | Write path | Read path | Cost behavior | Verdict |
|---|---|---|---|---|
| A: single premium KV | One path, no demotion | Low-latency KV GET | Cold checkpoints pay hot-tier GB-month | Simplest; fallback |
| B: hot KV + cold object | Batched checkpoints plus demotion job and manifest | Hot state from KV; cold via manifest | Cold GB-month at object rates | Winner for qualifying workloads |
| C: object-only | Object PUTs | Full object GET per step | Lowest per-GB cost | Breaks live-agent latency |
The table is the decision aid: choose Option B when the workload qualifies and batched writes are feasible; otherwise Option A keeps the system simpler without changing the read path.
Costs and numbers that matter
Cost per hour = (hot writes/hour × hot PUT price) + (hot GB-month × hot rate ÷ 730) + (cold GB-month × cold rate ÷ 730) + demotion job compute. Read each term as a separate meter: request charges scale with write rate, storage charges scale with retained bytes, and demotion compute scales with how often the job runs. Keeping the terms separate prevents a low storage rate from hiding a high request rate.
Run one consistency check on the checkpoint column before you accept it. Checkpoints every 30 minutes are 48 per day, or 1,440 across 30 days; 72 GB divided by 1,440 works out to about 50 MB per batched checkpoint. If your actual snapshot size is far from that, your cold-tier estimate is wrong and the arithmetic above does not transfer. The gap between the two checkpoints is $21.60 − $2.16 = $19.44, roughly 90% of this particular illustrative bill. Treat that as an upper bound and plan against a smaller figure, because replication inside the hot tier, minimum billable capacity, and per-request charges are not counted here.
The checklist for this fleet: confirm sustained writes exceed 10,000 per hour across a full week, not a burst; confirm more than 80% of keys are unread 24 hours after their last write; confirm the write path batches (at 14 writes per second, a one-second window produces roughly 14 writes per batch — if your client cannot coalesce, fix that first); confirm checkpoint cadence and snapshot size match the 48-per-day, ~50 MB assumption; confirm a restore from cold meets your recovery target; then re-run every rate in the ledger against your own contract before committing to the split. Because the guide states no grounding source supplies a 2026 tier price, the ledger is a formula to fill from your own bill, not a quote to trust.
Decision rules for the tiering call
Three if/then rules turn a week of write-rate and read-recency data into the tiering decision, and they are meant to be applied in order. Each one terminates the question: you either stay single-tier, split, or adjust a split you already have. The thresholds carry the decision; you do not need a cost estimate on the back of an envelope to get there.
If sustained writes stay below 10,000 per hour, stay single-tier. Below that rate, the demotion job and its manifest overhead outweigh the cold-rate saving they would recover — the split becomes machinery that earns nothing at low churn. Read "sustained" as the hourly median over the full week, not a burst. A fleet that spikes above the line twice a day and sits well under it otherwise is a single-tier fleet.
If sustained writes are at or above 10,000 per hour and more than 80% of bytes are unread after 24 hours, split — with a 24-hour demotion threshold and batched checkpoints. The 24-hour threshold follows from your own read-recency measurement: a key unread at that mark is cold by the data you collected, not by assumption. Batch the write path before you split, because demotion and promotion act on objects one at a time and an unbatched writer multiplies those operations.
If promotion rate exceeds 20% of keys per day, stop and fix the threshold instead. A key that returns inside a day was never cold, and moving it twice costs more than leaving it hot through both reads. Raise the demotion threshold for that key class, or keep the class hot, then re-measure before changing anything else.
Set the cutoffs from the observed distribution rather than from fixed guesses. The IBM Z Performance and Capacity Analytics documentation (June 2026 edition) reports workload delay against dynamic thresholds computed hourly for a given system, policy, and service class — the same discipline of deriving a bound from measured behavior and then monitoring against it, though that document carries no storage cost figure and cannot itself justify a tiering threshold. Re-run the comparison after any material change in step count or session length, since both shift write rate and read recency at once.
| Observed after one week | Decision | First check |
|---|---|---|
| Sustained writes below 10,000/hour | Stay single-tier | Median, not peak, across the week |
| Writes at or above 10,000/hour and more than 80% of bytes unread at 24 h | Split, 24-hour demotion, batched checkpoints | Whether the write path batches |
| Promotion above 20% of keys per day | Raise the threshold or keep the key hot | Which key classes keep returning |
What to do next
| Step | Action | Why it matters |
|---|---|---|
| 1 | Instrument write rate and read-recency per agent-state key for one full week before changing anything. Log every write and record whether each key is read within the 24-hour window. | This is the measurement the canonical decision rule depends on — without a week of per-key data you cannot tell whether you clear the 10,000 writes/hour bar or the 80% unread-after-24-hours bar. |
| 2 | Compare your measured write rate against the 10,000 writes/hour threshold and your unread-key share against the 80% threshold. If either falls short, stay single-tier. | The split only pays off when both conditions hold; splitting below either threshold adds tiering overhead without the ~60% saving. |
| 3 | If both thresholds are met, split into a hot KV tier plus a cold object tier and set the demotion threshold at 24 hours. | This is the exact configuration the decision rule prescribes — the 24-hour demotion matches the read-recency window you measured. |
| 4 | Batch the write path before you rely on the ~60% saving figure. | The ~60% total storage cost reduction applies only when writes are batched; an unbatched write path erodes the saving. |
| 5 | Re-check the comparison table above against your own workload, using the per-tier figures already shown there rather than re-deriving them. | Confirms the tiered hot KV plus cold object configuration beats keeping every write on a single premium tier at your measured write volume. |
| 6 | Re-run the one-week instrumentation after the split and confirm the unread-key share still clears the 80% bar at the 24-hour demotion threshold. | Read-recency drifts; a configuration that met the rule at rollout can fall back below the threshold and should revert to single-tier. |
Frequently Asked Questions
At what sustained write rate does tiering agent state become worthwhile?
Split into a hot KV tier plus a cold tier only if sustained writes exceed 10,000/hour and more than 80% of keys are unread after 24 hours.
How long should I instrument write rate and read-recency before splitting tiers?
Instrument write rate and read-recency per agent-state key for one week before splitting tiers.
What condition must the write path meet for the ~60% saving to actually apply?
The ~60% saving applies only when the write path is batched.
Where should cold checkpoints be stored in the tiered architecture?
Hot state lives on a low-latency KV store and cold checkpoints are demoted to object storage.
What do the HDD workload and failure-rate figures actually come from?
Both figures come from HDD workload and failure-rate definitions, not from agent-state benchmarks.
What savings range can hyperscaler exits deliver?
Hyperscaler exits can deliver 50–80% savings at real-world cost thresholds and workload signals.
Quick answers
| At what write rate should agent state be tiered to cut total storage cost by about 60%? | Tier agent state above roughly 10,000 writes per hour to cut total storage cost about 60%. |
| When does the ~60% saving from tiering apply? | The ~60% saving applies only when the write path is batched. |
| What should be instrumented before splitting tiers, and for how long? | Instrument write rate and read-recency per agent-state key for one week before splitting tiers. |
| What conditions must be met to split into a hot KV tier plus a cold tier? | Split into a hot KV tier plus a cold tier only if sustained writes exceed 10,000/hour and more than 80% of keys are unread after 24 hours. |
| What does the guide set as a concrete threshold for tiering agent state, and where does hot state live versus cold checkpoints? | This guide sets a concrete threshold for tiering agent state: about 10,000 writes per hour, with hot state on a low-latency KV store and cold checkpoints demoted to object storage. |
Also worth reading: GB200 NVL72 Checkpoints: When Async S3 Tiering Wins on Cost: GB200 NVL72 Checkpoints: When Async · Cross-Cloud Object Storage: Egress Math & ML Decision Framework: Cross-Cloud Object Storage: Egress Math · Object Storage P99 GET Latency: Why the Tail Is Topological: Object Storage P99 GET Latency: