# S3 vs MinIO PUT Latency at 500 TB: What the Benchmarks Hide

Wei Chen · August 29, 2026

> S3 vs MinIO PUT Latency at 500 TB: What the Benchmarks Hide. Meanwhile, S3 Express One Zone has already collapsed most of the latency...

| Takeaway | Detail |
| --- | --- |
| Write latency benchmarks mask the true TCO driver at scale | A 500 TB restore from S3 Glacier Instant Retrieval costs a significant amount in retrieval fees alone |
| Self-hosted NVMe clusters shift costs from egress to idle capacity | The same restore from a self-hosted MinIO cluster on NVMe costs nothing in egress but requires tens of thousands of dollars of raw NVMe capacity sitting idle between restores |
| Snapshot restoration overhead compounds storage economics | Large snapshots (1GB+) incur separate billing for snapshot storage distinct from compute or runtime costs |
| Platform teams optimize for cherry-picked metrics over operational reality | P99 Time-to-First-Byte (TTFT) is benchmarked with and without restored KV state in MinIO deployments while ignoring cross-replica routing rate and HBM eviction rate under heavy I/O loads |

Meanwhile, S3 Express One Zone has already collapsed most of the latency gap for workloads that actually demand low write times. The industry fixate on synthetic benchmarking cycles that measure cache hit rates by prefix or KV block, yet ignore that large snapshots (1GB+) incur separate billing for snapshot storage distinct from compute or runtime costs. True infrastructure planning must account for restore frequency, cross-region transfer latency, and the hidden toll of idle capacity rather than chasing millisecond improvements on hot paths.

S3 Standard's PUT latency is a function of its durability topology, not network distance. Every write triggers a quorum acknowledgment across a minimum of three Availability Zones before the client receives a 200 OK. This cross-AZ replication protocol forces observed PUT p50 to land at 40-80 ms from a same-region EC2 client, even on 2026-generation infrastructure where raw round-trip times are negligible. The latency you observe is the cost of the consensus mechanism; S3 buys 99.999999999% (11 nines) durability by design, distributing writes until multiple independent failure domains confirm receipt.

MinIO inverts this model by collapsing the quorum to the local site. A standard 16-drive erasure set with EC:4 (12 data, 4 parity) stripes objects directly to local NVMe over PCIe or NVMe-oF. The cluster acknowledges the PUT only after local stripe completion, bypassing inter-AZ replication traffic during the write path. MinIO's published warp benchmarks demonstrate p50 PUT performance in the 5-15 ms range on a single 4-node NVMe cluster. This topology tolerates up to 4 simultaneous drive failures but offers zero site loss protection compared to S3's multi-AZ spread. The latency gap is strictly the price of the topology, not lazy engineering; MinIO trades geographic redundancy for write responsiveness.

![Endless corridor brushed steel server racks stretching into](https://static.mm-ais.com/article-images-ai/s3-vs-minio-put-latency-at-500-tb-what-t-ai-391c7e04.jpg)
Endless corridor brushed steel server racks stretching into

## Two Write Paths

Benchmark numbers often mask the steady-state degradation inherent in NVMe controllers. Consumer- and datacenter-grade NVMe drives rely on SLC write caches to maintain low latency. Under sustained PUT loads, these caches exhaust rapidly. Once the cache is full, write amplification spikes, pushing p99 latency from ~10 ms to 30-60 ms as the controller manages garbage collection and wear leveling. According to "When Repeated Context Becomes an Infrastructure Problem | MinIO", HBM eviction rates must be monitored to prevent performance degradation under heavy NVMe I/O loads, and cross-replica routing rates are measured to assess consistency. In practice, MinIO's benchmark figures represent cache-warm states unless the test wrote past the cache threshold, typically exceeding 2-3x the total drive capacity sequentially. For workloads approaching 500 TB, steady-state p99 behavior dictates whether the 5-10x latency advantage holds or degrades toward the S3 Standard floor.

The hardware comparison gap remains structural: S3 cannot be benchmarked on identical silicon. StorageReview's independent 2024 object-storage benchmark compared MinIO against Ceph on identical NVMe arrays, where MinIO posted roughly 2-3x Ceph's PUT throughput. This stands as the only widely cited third-party hardware-controlled comparison, yet it excludes S3 entirely because the managed service operates on proprietary infrastructure. For self-hosted deployments, this confirms MinIO's efficiency advantage over alternative open-source stacks on commodity NVMe, but it does not validate the latency delta against S3 Express, which requires direct cloud-side measurement.

At 500 TB in 2026, the decision matrix collapses to a single operational constraint: you are either buying latency or paying for retrieval. The comparison below maps the three viable architectures against the four dimensions that dictate total cost of ownership and risk exposure. S3 Standard remains the baseline for durability and restore economics; S3 Express One Zone offers a cloud-native path to sub-10 ms PUT p99 at the cost of single-AZ isolation; MinIO on NVMe delivers comparable tail-latency but forces you to absorb hardware depreciation, erasure coding overhead, and full operational responsibility.

The winner for latency-sensitive workloads is S3 Express One Zone, provided your architecture can tolerate single-AZ risk. It delivers sub-10 ms PUT p99 performance that rivals MinIO's single-digit millisecond tail, yet it eliminates every hardware operation from your runbook. You get the latency profile without managing NVMe wear, drive failures, or cluster rebalancing. MinIO only wins this row when the data must physically stay on-premises due to sovereignty constraints; otherwise, the cloud-native option dominates on operational simplicity.

| Write Path | Quorum Scope | PUT p50 (Warm) | Durability Model | Steady-State Risk |
| --- | --- | --- | --- | --- |
| S3 Standard | 3+ AZs | 40-80 ms | 11 Nines (Cross-AZ) | N/A (Managed) |
| MinIO EC:4 | Local Site | 5-15 ms | 4 Drive Failures / Zero Site Loss | p99 → 30-60 ms post-cache |
| S3 Express OZ | Single AZ |

Canonical: https://x-oss.com/blog/s3-vs-minio-put-latency-at-500-tb-what-the-benchmarks-hide.php
Markdown: https://x-oss.com/blog/s3-vs-minio-put-latency-at-500-tb-what-the-benchmarks-hide.php/index.md