Description
Checkboxes for prior research
- I've gone through Developer Guide and API reference
- I've checked AWS Forums and StackOverflow.
- I've searched for previous similar issues and didn't find any solution.
Describe the bug
When using the browser version of the SDK, HTTP requests are made with fetch
which inherently decompresses responses (see the spec) before returning them. As a result, when making an S3 GetObject
request for a compressed file, the response body that the SDK sees is uncompressed but it still expects the checksum of that uncompressed response body to match the checksum in S3 (which is the checksum of the compressed file). Thus, the SDK reports a checksum mismatch and throws an error.
Regression Issue
- Select this option if this issue appears to be a regression.
SDK version number
@aws-sdk/client-s3@3.758.0
Which JavaScript Runtime is this issue in?
NodeJS / workerd / Browser
Details of the browser/Node.js/ReactNative version
The reproduction is using Node v20.10.0, but I originally encountered this bug using workerd v1.20241202.0
Reproduction Steps
Here is a repository with example code to reproduce the bug: https://github.com/sdeneen/s3-js-sdk-checksum-bug
It tries and fails to get a compressed file from a public S3 bucket.
The README includes instructions on how to build/run it. Note that this is specific to the browser implementation of the SDK. The example runs in Node but is configured to bundle the browser implementation.
Observed Behavior
$ node dist/s3.mjs
Error: Checksum mismatch: expected "uboIqQ==" but received "xrd+Yw==" in response header "x-amz-checksum-crc32".
at Object.flush (file:///Users/sdeneen/Repos/s3-js-sdk-checksum-bug/dist/s3.mjs:692:31)
The former checksum is that of the compressed file, and the latter is the checksum of the uncompressed file.
Expected Behavior
The file should be successfully fetched from S3, without resulting in a checksum mismatch error.
Possible Solution
No response
Additional Information/Context
No response