Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 29, 2025

S3 backups fail when proxy is enabled, producing incomplete uploads with error put_object write size < data.size(), w_size=15728640, data.size=16396159. AWS SDK v3 uses the global fetch/undici dispatcher which routes through the proxy manager, causing stream corruption on large files.

Changes

Configure S3Client to bypass proxy

  • Add direct undici.Agent dispatcher to S3Storage constructor
  • Configure S3Client with custom FetchHttpHandler using direct dispatcher
  • Set 60s connection timeout and 5min request timeout for large file handling
// S3Storage.ts
const directDispatcher = new UndiciAgent({
  connect: { timeout: 60000 }
})

const requestHandler = new FetchHttpHandler({
  requestTimeout: 300000,
  dispatcher: directDispatcher
})

this.client = new S3Client({
  // ... existing config
  requestHandler
})

This ensures S3 requests bypass the global proxy, preventing stream interference during large file uploads while maintaining proxy functionality for other services.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.sheetjs.com
    • Triggering command: /usr/local/bin/node /home/REDACTED/work/cherry-studio/cherry-studio/.yarn/releases/yarn-4.9.1.cjs install (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[Bug]: S3 Backup Failed</issue_title>
<issue_description>> [!NOTE]

This issue/comment/review was translated by Claude.

Issue Checklist

  • I understand that issues are for feedback and problem solving, not for complaining in the comment section, and will provide as much information as possible to help solve the problem.
  • My issue is not listed in the FAQ.
  • I've looked at pinned issues and searched for existing Open Issues, Closed Issues, and Discussions, no similar issue or discussion was found.
  • I've filled in short, clear headings so that developers can quickly identify a rough idea of what to expect when flipping through the list of issues. And not "a suggestion", "stuck", etc.
  • I've confirmed that I am using the latest version of Cherry Studio.

Platform

macOS

Version

v1.6.5

Bug Description

S3-compatible storage backup failed, as shown in the image below

Image

Steps To Reproduce

Managing backups can retrieve normally, which indicates my S3 configuration is correct

Image

Clicking "Backup Now" shows backup failed

Image

Expected Behavior

The expected behavior is that backup should succeed normally

Relevant Log Output

es-Rilh_bH0.js:85 <error> [BackupService] backupToS3: Error uploading file to S3: Error: Error invoking remote method 'backup:backupToS3': InternalError: Io error: put_object write size < data.size(), w_size=15728640, data.size=16396159
processLog @ es-Rilh_bH0.js:85
error @ es-Rilh_bH0.js:116
backupToS3 @ index-CvOKVPUf.js:20853
await in backupToS3
handleBackup @ index-CvOKVPUf.js:110152
handleOk @ store-DHkT-mYs.js:75219
(anonymous) @ ImageViewer-CZSiC8u0.js:4963
processDispatchQueue @ client-DDbmooWc.js:6971
(anonymous) @ client-DDbmooWc.js:7257
batchedUpdates$1 @ client-DDbmooWc.js:1188
dispatchEventForPluginEventSystem @ client-DDbmooWc.js:7047
dispatchEvent @ client-DDbmooWc.js:8605
dispatchDiscreteEvent @ client-DDbmooWc.js:8587
index-CvOKVPUf.js:110160 Uncaught (in promise) Error: Error invoking remote method 'backup:backupToS3': InternalError: Io error: put_object write size < data.size(), w_size=15728640, data.size=16396159
Promise.then
backupToS3 @ index-CvOKVPUf.js:20797
await in backupToS3
handleBackup @ index-CvOKVPUf.js:110152
handleOk @ store-DHkT-mYs.js:75219
(anonymous) @ ImageViewer-CZSiC8u0.js:4963
processDispatchQueue @ client-DDbmooWc.js:6971
(anonymous) @ client-DDbmooWc.js:7257
batchedUpdates$1 @ client-DDbmooWc.js:1188
dispatchEventForPluginEventSystem @ client-DDbmooWc.js:7047
dispatchEvent @ client-DDbmooWc.js:8605
dispatchDiscreteEvent @ client-DDbmooWc.js:8587
es-Rilh_bH0.js:85 <error> [BackupService] backupToS3: Error uploading file to S3: Error: Error invoking remote method 'backup:backupToS3': InternalError: Io error: put_object write size < data.size(), w_size=15728640, data.size=16396159
processLog @ es-Rilh_bH0.js:85
error @ es-Rilh_bH0.js:116
backupToS3 @ index-CvOKVPUf.js:20853
await in backupToS3
handleBackup @ index-CvOKVPUf.js:110152
handleOk @ store-DHkT-mYs.js:75219
(anonymous) @ ImageViewer-CZSiC8u0.js:4963
processDispatchQueue @ client-DDbmooWc.js:6971
(anonymous) @ client-DDbmooWc.js:7257
batchedUpdates$1 @ client-DDbmooWc.js:1188
dispatchEventForPluginEventSystem @ client-DDbmooWc.js:7047
dispatchEvent @ client-DDbmooWc.js:8605
dispatchDiscreteEvent @ client-DDbmooWc.js:8587
index-CvOKVPUf.js:110160 Uncaught (in promise) Error: Error invoking remote method 'backup:backupToS3': InternalError: Io error: put_object write size < data.size(), w_size=15728640, data.size=16396159

Additional Context

No response


Original Content

Issue Checklist

  • I understand that issues are for feedback and problem solving, not for complaining in the comment section, and will provide as much information as possible to help solve the problem.
  • My issue is not listed in the FAQ.
  • I've looked at pinned issues and searched for existing Open Issues, Closed Issues, and Discussions, no sim...

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits October 29, 2025 07:29
Co-authored-by: GeorgeDong32 <98630204+GeorgeDong32@users.noreply.github.com>
Co-authored-by: GeorgeDong32 <98630204+GeorgeDong32@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix S3 backup failure issue Oct 29, 2025
Copilot AI requested a review from GeorgeDong32 October 29, 2025 07:37
Copilot finished work on behalf of GeorgeDong32 October 29, 2025 07:37
@GeorgeDong32 GeorgeDong32 changed the title Fix S3 backup failure with proxied connections Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants