fix: repair classic xref tables whose entries end with a bare LF - #474
fix: repair classic xref tables whose entries end with a bare LF#474xenokratos wants to merge 2 commits into
Conversation
5181f89 to
762e02d
Compare
Some PDF writers terminate classic cross-reference entries with a single byte (a bare LF) instead of the two-byte end-of-line the spec requires, making each entry 19 bytes instead of exactly 20. The offsets inside such a table are all correct - only the stride is wrong - but the file fails every entry point with `invalid file trailer`, while pypdf, pdfium and pdfjs all read it. Hit in the wild on a 65MB, 29-page production document. Adds a repair candidate alongside the existing startxref-pointer recovery (firecrawl#230): rebuild the table at its original position with spec 20-byte entries and leave every other byte untouched. The startxref pointer still lands on the table start, and object offsets point backwards at content that has not moved. Returns None when every entry already has the spec stride, so well-formed files never produce a useless candidate. Fixture is synthetic: two byte-identical PDFs except the xref entry EOL (SP LF vs bare LF); the bare-LF variant reproduced the failure verbatim before this change. 1199 existing tests pass unchanged.
762e02d to
26de46b
Compare
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/integration_tests.rs">
<violation number="1" location="tests/integration_tests.rs:4403">
P3: The `/// Regression for #228` doc comment (about a corrupted `startxref` pointer) now sits directly above the new `test_process_pdf_recovers_bare_lf_xref_entries` test, which is about a different issue (#473 / bare-LF xref stride). The new test was inserted between that comment and the actual #228 test, `test_process_pdf_recovers_corrupted_startxref_pointer`, so the comment misattributes the regression it describes. Move the new test below the #228 block (or move the comment) so each test keeps its own doc.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Fix all with cubic | Re-trigger cubic
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/lib.rs">
<violation number="1" location="src/lib.rs:4030">
P2: When a classic table mixes bare-LF entries with normal `SP CR LF` entries, this arm consumes only `SP CR` and the next iteration starts on `LF`, so normalization bails out and the PDF remains unrepairable. Consume three bytes for `SP CR LF` before the two-byte `SP CR`/`SP LF` cases.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
The two-byte arm matched `SP CR` inside a `SP CR LF` terminator and left the next entry starting on the stray `LF`, so a table mixing bare-LF entries with `SP CR LF` ones bailed out and the file stayed unrepairable. Matching the three-byte form first fixes that, and treating any non-two-byte EOL as malformed also covers a table that is uniformly `SP CR LF` (21-byte stride), which failed the same way. Fixture mixes both strides in one table. Moves the bare-LF test below the firecrawl#228 block so that issue's doc comment stays attached to its own test.
There was a problem hiding this comment.
0 issues found across 3 files (changes from recent commits).
Shadow auto-approve: would auto-approve. Adds a repair candidate that rebuilds the last classic xref table in place when its EOL bytes don't match the spec, fixing 'invalid file trailer' for bare-LF and mixed-EOL xref tables; confined to malformed input and covered by new tests.
Re-trigger cubic
Fixes #473.
Adds a repair candidate alongside #230's startxref-pointer recovery: when the last classic xref table's entries end with a single EOL byte (19-byte stride) instead of the spec's two (exactly 20), rebuild the table at its original position with spec
SP LFentries and leave every other byte untouched. Thestartxrefpointer still lands on the table start, and object offsets point backwards at content that has not moved.Returns
Nonewhen every entry already has the spec stride, so well-formed files never produce a useless repair candidate — and any parse ambiguity inside the table (non-digit offsets, unknown entry kind, missing EOL) also bails toNonerather than guessing, since this crate processes untrusted input.The fixture is fully synthetic (generated, one page, one text line). The bare-LF variant reproduced the failure verbatim before this change; on a real 65MB / 29-page production document with this malformation, extraction goes from
invalid file trailerto full text in 0.13s. 1199 existing tests pass unchanged.Summary by cubic
Fixes #473. Classic xref entries whose EOL isn't the spec's two bytes — a bare
LF(19-byte stride) orSP CR LF(21), including tables mixing both — previously failed withinvalid file trailer; they now parse because the last classic xref table is rebuilt in place with spec 20-byteSP LFentries. Well-formed files are unaffected, and ambiguous tables are left alone rather than guessed at.LF.Nonewhen every entry already has the spec stride, so well-formed files never produce a useless repair candidate; parse ambiguity inside the table also bails toNone.Written for commit 69827c8. Summary will update on new commits.