Github Pages sends invalid responses to range requests

[llimllib]

Select Topic Area

Bug

Body

Users of duckdb-wasm currently cannot use github pages on firefox, because of an invalid response of the github pages server to range requests.

The issue is that github pages is returning a range of the gzipped file, which is not itself a valid gzip stream, causing decoding to fail in Firefox. Chrome and Safari don't send accept-encoding: gzip, so github pages does not return invalid data to them.

You can reproduce easily with curl:

# works
$ curl 'https://llimllib.github.io/nba_data/gamelogs.parquet' -H 'Range: bytes=100-140' | xxd   
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    41  100    41    0     0    375      0 --:--:-- --:--:-- --:--:--   376
00000000: 470d 100d 0804 0048 0d10 0d08 0400 490d  G......H......I.
00000010: 100d 0804 004a 0d10 0d08 0400 4b0d 100d  .....J......K...
00000020: 0804 004c 0d10 0d08 04                   ...L.....

# fails
$ curl 'https://llimllib.github.io/nba_data/gamelogs.parquet' --compressed -H 'Range: bytes=100-140' | xxd
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0    41    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (61) Error while processing content unencoding: incorrect header check

# if you request the first bytes of the file, you see the magic gzip header 1f8b
$ curl -s 'https://llimllib.github.io/nba_data/gamelogs.parquet' -H 'accept-encoding: gzip' -H 'Range: bytes=0-8' | xxd                   
00000000: 1f8b 0800 0000 0000 00                   .........'

# if you request bytes 2-8, you see that pages just slices the gzip response, which is invalid gzip
$ curl -s 'https://llimllib.github.io/nba_data/gamelogs.parquet' -H 'accept-encoding: gzip' -H 'Range: bytes=2-8' | xxd
00000000: 0800 0000 0000 00                        .......

$ curl -s 'https://llimllib.github.io/nba_data/gamelogs.parquet' -H 'accept-encoding: gzip' -H 'Range: bytes=2-8' | gunzip
gunzip: unknown compression format

cf duckdb/duckdb-wasm#1932
cf bugzilla 1874840
cf observablehq/framework#1886
cf apple/embedding-atlas#20
cf protomaps/PMTiles#584 (comment)

related (but not duplicate) discussion: https://github.com/orgs/community/discussions/162857

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[carlopi]

Also somewhat relevant the bugzilla report, in particular the 9th comment: https://bugzilla.mozilla.org/show_bug.cgi?id=1874840#c9.

[llimllib]

I filed a PR on mime-db to mark parquet files as incompressible, which wouldn't fix the github pages bug but would un-block users of observable framework.

(Also I think it's the correct behavior as ~all parquet files are already compressed)

jshttp/mime-db#411