From the course: AWS Certified Data Engineer Associate (DEA-C01) Cert Prep

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Amazon S3 performance optimization

Amazon S3 performance optimization

From the course: AWS Certified Data Engineer Associate (DEA-C01) Cert Prep

Start my 1-month free trial Buy for my team

Amazon S3 performance optimization

“

(soothing music) - Hi guys. In this lesson I'm going to cover a quick bit of theory on performance optimization in relation to Amazon S3. So let's look at some design patterns for optimizing S3 Performance. S3 will automatically scale to very high request rates with at least, and this is per prefix, 3,500 PUT, COPY, POST or DELETE requests a second and 5,500 GET/HEAD requests a second. Now you can increase the read and write performance by using parallelization across multiple prefixes. So you're going to spread your data across more prefixes, and these limitations only apply within each prefix. To increase upload speeds over long distances, you can use S3 Transfer Acceleration that leverages the CloudFront network to upload your data and move it into S3 across the AWS Global Network. You can also use what are called Byte-Range fetches with the Range HTTP header, and that means that you are only transferring the specified Byte-Range from an object that you actually need. You should…

Contents