AWS Storage

AWS S3storageAmazon S3 (Simple Storage Service) is a scalable object storage service provided by Amazon Web Services (AWS). S3 Transfer AccelerationstorageAmazon S3 Transfer Acceleration is a feature that enables fast, easy, and secure transfers of files over long…

This section covers Amazon S3, the object store at the centre of most AWS data architectures, and S3 Transfer Acceleration, which speeds up long-distance uploads and downloads. The overview below places S3 among the other AWS storage services so it is clear when object storage is the right choice.

Object storage and the transfer behaviour that determines how fast data gets in and out.



‹ AWS


1. How S3 is used

S3 in a data platform Clients worldwide SDK, CLI, apps Edge location Transfer Acceleration (optional) S3 bucket in a region objects, versions, lifecycle rules Athena / Glue EMR / Spark Lambda events Redshift Spectrum

S3 stores objects (bytes plus metadata) under keys in buckets that live in one region and are replicated across availability zones. Analytics services read the data in place, which is what makes S3 the storage layer of an AWS data lake. Transfer Acceleration routes traffic from clients far from the bucket's region into the nearest CloudFront edge location and then over the AWS backbone, which can help large transfers over long distances; for clients in the same region it adds cost without benefit.

2. Choosing a storage service

ServiceTypeUse when
Amazon S3Object storage, HTTP APIData lakes, backups, static assets, any scale
Amazon EBSBlock volume for one EC2 instanceDatabases and boot volumes needing low latency
Amazon EFSShared NFS file systemMany instances need a POSIX file system
S3 Glacier classesArchive tiers of S3Rarely accessed data, retrieval delay acceptable

Within S3, storage classes (Standard, Intelligent-Tiering, Standard-IA, Glacier tiers) trade storage price against retrieval cost and latency. Lifecycle rules can move objects between classes automatically as they age.

Amazon AthenaanalyticsSQL over S3 Amazon AWSSectionall AWS sections Apache Parquetstorage formatsthe usual file format on S3