Skip to content

Checksum mismatch error fetching compressed files from S3 (browser SDK) #6910

Open
@sdeneen

Description

@sdeneen

Checkboxes for prior research

Describe the bug

When using the browser version of the SDK, HTTP requests are made with fetch which inherently decompresses responses (see the spec) before returning them. As a result, when making an S3 GetObject request for a compressed file, the response body that the SDK sees is uncompressed but it still expects the checksum of that uncompressed response body to match the checksum in S3 (which is the checksum of the compressed file). Thus, the SDK reports a checksum mismatch and throws an error.

Regression Issue

  • Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/client-s3@3.758.0

Which JavaScript Runtime is this issue in?

NodeJS / workerd / Browser

Details of the browser/Node.js/ReactNative version

The reproduction is using Node v20.10.0, but I originally encountered this bug using workerd v1.20241202.0

Reproduction Steps

Here is a repository with example code to reproduce the bug: https://github.com/sdeneen/s3-js-sdk-checksum-bug

It tries and fails to get a compressed file from a public S3 bucket.

The README includes instructions on how to build/run it. Note that this is specific to the browser implementation of the SDK. The example runs in Node but is configured to bundle the browser implementation.

Observed Behavior

$ node dist/s3.mjs
Error: Checksum mismatch: expected "uboIqQ==" but received "xrd+Yw==" in response header "x-amz-checksum-crc32".
    at Object.flush (file:///Users/sdeneen/Repos/s3-js-sdk-checksum-bug/dist/s3.mjs:692:31)

The former checksum is that of the compressed file, and the latter is the checksum of the uncompressed file.

Expected Behavior

The file should be successfully fetched from S3, without resulting in a checksum mismatch error.

Possible Solution

No response

Additional Information/Context

No response

Metadata

Metadata

Labels

bugThis issue is a bug.p2This is a standard priority issuethird-partyThis issue is related to third-party libraries or applications.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions