S3 Is the Future, S3 Is the Past
S3's dominance in cloud architecture is built on hard disk assumptions that are now obsolete. SSDs offer 100-microsecond latency and millions of IOPS, making caching, batching, and separate metadata stores unnecessary. Yet cloud providers have little incentive to cannibalize their revenue, so we must build the SSD-based disaggregated storage ourselves.
The irony is that S3-based architecture is becoming the industry standard at precisely the time when the hardware constraints that motivated it are fading.