Fix PDF linear-scan crash and add configurable render background

.cspell.yaml

@@ -41,6 +41,8 @@ words:
   - rasterizing
   - startxref
   - Startxref
+  - xstartxref
+  - mystartxref
   - SYNT
   - undecoded
   - BITMAPCOREHEADER
@@ -302,6 +304,8 @@ words:
   - decryptor
   - Decryptor
   - keystream
+  - qpdf
+  - mutool
 
 # Exclude common build artifacts, dependencies, and vendored third-party code
 ignorePaths:

.reviewmark.yaml

@@ -763,6 +763,7 @@ reviews:
       - "src/DemaConsulting.CanvasNet.Pdf/PdfDocument.FontFallback.cs"  # phase 6 unit source (font fallback)
       - "src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Text.cs"          # phase 4 unit source (text rendering)
       - "src/DemaConsulting.CanvasNet.Pdf/PdfPageInfo.cs"              # phase 1 unit source (supporting record struct)
+      - "src/DemaConsulting.CanvasNet.Pdf/PdfRenderOptions.cs"         # supporting options type (background color)
       - "src/DemaConsulting.CanvasNet.Pdf/NamespaceDoc.cs"             # Pdf namespace documentation
       - "test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs"  # unit tests
       - "test/DemaConsulting.CanvasNet.Pdf.Tests/PdfSystemIntegrationTests.cs"  # integration tests

docs/design/canvas-net-pdf/pdf-document.md

@@ -227,6 +227,15 @@ re-parsing it each time — a property a purely static API could not express.
   sufficient to size a `Render` call, because a later phase's content-stream-to-device
   transformation matrix construction needs the effective rotation value itself, not just its
   width/height side effect.
+- **`PdfRenderOptions`** (public, `sealed class`, `PdfRenderOptions.cs`) — a small, deliberately
+  growable bag of page-rendering configuration, documented inline here for the same reason as
+  `PdfPageInfo` above. Currently exposes a single `init`-only `BackgroundColor` property
+  (`Canvas.Rgba32`, defaulting to opaque white) plus a `static readonly Default` instance used
+  whenever a `Render` caller passes `null`. **Design decision**: a `sealed class` with `init`
+  properties was chosen over a `record`/`record struct` specifically so future properties can be
+  added without a breaking positional-argument/constructor change for any caller already using
+  `new PdfRenderOptions { ... }` object-initializer syntax - no specific future property is
+  planned or implied by this shape choice.
 
 ### Key Methods
 
@@ -254,16 +263,26 @@ re-parsing it each time — a property a purely static API could not express.
 - **`GetPageInfo(int pageIndex)`** — disposed-check, then `ArgumentOutOfRangeException` for
   `pageIndex < 0 || pageIndex >= PageCount`, then returns `_pages[pageIndex]` (already computed
   during `Open`).
-- **`Render(int pageIndex, int width, int height)`** — disposed-check and `pageIndex`
-  range-check identical to `GetPageInfo`, then builds `new Surface(width, height)`, resolves the
-  page's leaf node, raw `/MediaBox` origin, and inherited `/Resources` via `ResolvePageDetails`,
-  derives the page's base CTM via `BuildBaseCtm` (see _Content-Stream Interpreter_ below),
-  resolves the page's `/Contents` bytes via `ResolvePageContentBytes`, executes them via
-  `ExecuteContentStream` (passing the resolved `/Resources`), and returns the painted surface.
-  `Surface`'s own constructor supplies the `width`/`height` `ArgumentOutOfRangeException`/
-  `MaxDimension` contract; this is deliberately not duplicated here, and the caller-specified
-  `width`/`height` is used exactly as given — it is never clamped to, or derived from, the page's
-  own `/MediaBox` size.
+- **`Render(int pageIndex, int width, int height, PdfRenderOptions? options = null)`** —
+  disposed-check and `pageIndex` range-check identical to `GetPageInfo`, then builds `new
+  Surface(width, height)` and immediately clears it via `surface.Clear((options ??
+  PdfRenderOptions.Default).BackgroundColor)` (opaque white when `options` is `null` or leaves
+  `BackgroundColor` at its own default), resolves the page's leaf node, raw `/MediaBox` origin,
+  and inherited `/Resources` via `ResolvePageDetails`, derives the page's base CTM via
+  `BuildBaseCtm` (see _Content-Stream Interpreter_ below), resolves the page's `/Contents` bytes
+  via `ResolvePageContentBytes`, executes them via `ExecuteContentStream` (passing the resolved
+  `/Resources`), and returns the painted surface. `Surface`'s own constructor supplies the
+  `width`/`height` `ArgumentOutOfRangeException`/`MaxDimension` contract; this is deliberately not
+  duplicated here, and the caller-specified `width`/`height` is used exactly as given — it is
+  never clamped to, or derived from, the page's own `/MediaBox` size. A caller that needs the
+  previous fully transparent background back (for example to composite the result over
+  something else itself) passes `new PdfRenderOptions { BackgroundColor = new(0, 0, 0, 0) }`.
+- **`Render(int pageIndex, float dpi, PdfRenderOptions? options = null)`** — disposed-check and
+  `pageIndex` range-check identical to `GetPageInfo`, then `ArgumentOutOfRangeException` for a
+  non-positive or non-finite `dpi`, computes `width`/`height` by scaling `GetPageInfo(pageIndex)`'s
+  own point-sized dimensions by `dpi / 72.0` (rounding away from zero), and forwards unchanged to
+  `Render(int, int, int, PdfRenderOptions?)` above — `options` (and its `BackgroundColor`
+  default/override semantics) is simply passed through as-is.
 - **`Dispose()`** — idempotent (mirrors `Surface`'s exact pattern): `if (_disposed) return;` then
   `_disposed = true;`. No finalizer (only managed memory — the buffered bytes and parsed object
   model — is held).
@@ -478,8 +497,9 @@ its own distinguishable `Feature` token.
   `/Contents`: a single stream is decoded directly via `GetStreamDecodedBytes`; an array of
   streams is decoded entry-by-entry and concatenated with a single space byte inserted between
   each entry (per the PDF specification's own requirement, so adjacent tokens from different
-  streams can never merge); a page with no `/Contents` key returns an empty array (rendered as a
-  blank page, exactly as every page did in Phase 1).
+  streams can never merge); a page with no `/Contents` key returns an empty array (rendered as
+  just its `PdfRenderOptions.BackgroundColor`-cleared surface, with no further geometry painted
+  over it).
 - **Graphics-state operators (`PdfDocument.GraphicsState.cs`)** — `OpPushGraphicsState`
   (`q`, pushes a clone of `_gs`), `OpPopGraphicsState` (`Q`, pops into `_gs`, tolerating an empty
   stack as a documented no-op leniency distinct from this class's strict operand-count/type

docs/reqstream/canvas-net-pdf.yaml

@@ -104,7 +104,7 @@ sections:
           - CanvasNetPdf-PdfDocument-Render
           - CanvasNetPdf-PdfDocument-RenderWithDpi
         tests:
-          - CanvasNetPdf_SystemIntegration_PdfRender_ReturnsBlankSizedSurface
+          - CanvasNetPdf_SystemIntegration_PdfRender_ReturnsOpaqueWhiteSizedSurface
           - CanvasNetPdf_SystemIntegration_PdfUnsupportedFormatValidation_MalformedContentStreamThrowsInvalidDataException
 
       - id: CanvasNetPdf-Dispose

docs/reqstream/canvas-net-pdf/pdf-document.yaml

@@ -69,6 +69,8 @@ sections:
               and must be supported for broad real-world document compatibility.
             tests:
               - PdfDocument_Open_ClassicXref_ResolvesRootAndPage
+              - PdfDocument_Open_ClassicXref_EntryOffsetBeyondInt32Range_ThrowsInvalidDataException
+              - PdfDocument_Open_ClassicXref_EntryPointingAtWrongObjectHeader_ThrowsInvalidDataException
 
           - id: CanvasNetPdf-PdfDocument-XrefStream
             title: >-
@@ -118,6 +120,30 @@ sections:
               well-formed documents still be opened instead of always failing outright.
             tests:
               - PdfDocument_Open_MalformedStartxref_FallsBackToLinearScan
+              - PdfDocument_Open_LinearScanFallback_ObjMarkerInsideStreamPayload_DoesNotCorruptOffsets
+              - PdfDocument_Open_Encrypted_LinearScanFallback_RecoversObjectsCompressedInObjectStream
+              - PdfDocument_Open_LinearScanFallback_StreamWordInsideDictionaryValue_DoesNotSkipFollowingObject
+              - PdfDocument_Open_LinearScanFallback_TrailerKeywordInsideStreamPayload_DoesNotReplaceRealTrailer
+              - PdfDocument_Open_LinearScanFallback_ObjMarkerInsideNonStreamDictionaryValue_DoesNotCorruptOffset
+              - PdfDocument_Open_LinearScanFallback_EarlyFalseEndstreamInPayload_DoesNotCorruptOffsets
+              - PdfDocument_Open_LinearScanFallback_TrailerKeywordInCommentOutsideStream_DoesNotReplaceRealTrailer
+              - PdfDocument_Open_LinearScanFallback_EarlyFalseEndstreamWithIndirectLength_DoesNotCorruptOffsets
+              - PdfDocument_Open_LinearScanFallback_ObjectHeaderNotAtLineStart_StillResolves
+              - PdfDocument_Open_LinearScanFallback_ObjectHeaderDecoyInsideMultilineString_DoesNotCorruptOffsets
+              - PdfDocument_Open_LinearScanFallback_UnsupportedFilterObjectStream_DoesNotAbortRecovery
+              - PdfDocument_Open_LinearScanFallback_TrailerKeywordAsNameToken_DoesNotReplaceRealTrailer
+              - PdfDocument_Open_LinearScanFallback_ManyStreamPayloadRangesBeforeTrailer_StillResolvesTrailer
+              - PdfDocument_Open_LinearScanFallback_TrailerKeywordAfterHashByte_DoesNotReplaceRealTrailer
+              - PdfDocument_Open_LinearScanFallback_ObjectNumberAsTokenSuffix_DoesNotCorruptOffset
+              - PdfDocument_Open_LinearScanFallback_ObjectNumberAsNameToken_DoesNotCorruptOffset
+              - PdfDocument_Open_LinearScanFallback_SecondAngleBracketAsHexStringStart_DoesNotCorruptOffset
+              - PdfDocument_Open_LinearScanFallback_LongUnterminatedLiteralStringRun_CompletesPromptly
+              - PdfDocument_Open_ClassicXref_StartxrefAsTokenSuffix_DoesNotRedirectToBogusOffset
+              - PdfDocument_Open_ClassicXref_StartxrefAsNameToken_DoesNotRedirectToBogusOffset
+              - PdfDocument_Open_LinearScanFallback_CommentBeforeEndobj_DoesNotCorruptOffsets
+              - PdfDocument_Open_LinearScanFallback_CommentInObjectHeader_ResolvesObject
+              - PdfDocument_Open_LinearScanFallback_CommentBeforeDeclaredLengthEndstream_DoesNotCorruptOffsets
+              - PdfDocument_Open_LinearScanFallback_UnterminatedCommentAtForwardScanBound_DoesNotCorruptOffsets
 
           - id: CanvasNetPdf-PdfDocument-PageTreeTraversal
             title: >-
@@ -326,19 +352,27 @@ sections:
               The PdfDocument class shall return a Surface of the caller-specified width and
               height for a valid page index, painting it with the page's interpreted content-
               stream geometry (see CanvasNetPdf-PdfDocument-ContentStreamDispatch and its sibling
-              requirements below) or, when the page declares no /Contents at all, a fully
-              transparent surface; it shall validate the page index the same way GetPageInfo does,
-              and propagate Surface's own width/height validation unwrapped rather than
-              duplicating it.
+              requirements below) over a surface cleared to the caller-configurable
+              PdfRenderOptions.BackgroundColor (opaque white by default) via an optional trailing
+              PdfRenderOptions? options parameter; it shall validate the page index the same way
+              GetPageInfo does, and propagate Surface's own width/height validation unwrapped
+              rather than duplicating it.
             justification: |
               Phase 1 established Render's final call shape (page index plus caller-chosen output
               size) so Phase 2's real content-stream rendering could be added without a breaking
-              API change; callers need a correctly sized, correctly painted result.
+              API change; callers need a correctly sized, correctly painted result, and many
+              downstream consumers (for example PDF-to-image conversion for print or preview)
+              expect an opaque, non-transparent page background by default while still being able
+              to opt back into the old fully transparent behavior when they need to composite the
+              result over something else themselves.
             tests:
-              - PdfDocument_Render_ValidPageIndex_ReturnsCorrectlySizedBlankSurface
+              - PdfDocument_Render_ValidPageIndex_ReturnsCorrectlySizedOpaqueWhiteSurface
               - PdfDocument_Render_OutOfRangePageIndex_ThrowsArgumentOutOfRangeException
               - PdfDocument_Render_InvalidWidth_PropagatesSurfaceArgumentOutOfRangeException
-              - PdfDocument_ContentStream_NoContents_RendersBlankSurface
+              - PdfDocument_ContentStream_NoContents_RendersWithoutThrowing
+              - PdfDocument_Render_NoOptions_DefaultsToOpaqueWhiteBackground
+              - PdfDocument_Render_CustomBackgroundColor_ClearsSurfaceToThatColor
+              - PdfDocument_Render_TransparentBackgroundColor_ReproducesOldFullyTransparentBehavior
 
           - id: CanvasNetPdf-PdfDocument-RenderWithDpi
             title: >-
@@ -347,8 +381,9 @@ sections:
               compute an output width/height in pixels themselves, scaling the page's own
               rotation-adjusted point-space size by dpi/72, preserving its aspect ratio, and
               rounding the result to the nearest pixel before producing the same painted Surface
-              as the width/height overload; it shall reject a dpi that is not a positive, finite
-              number, and validate the page index the same way the width/height overload does.
+              as the width/height overload (forwarding its own optional PdfRenderOptions?
+              parameter unchanged); it shall reject a dpi that is not a positive, finite number,
+              and validate the page index the same way the width/height overload does.
             justification: |
               Callers frequently think in terms of a target resolution (dots per inch) rather than
               an absolute pixel size, and computing a page's correctly aspect-ratio-preserving
@@ -360,6 +395,7 @@ sections:
               - PdfDocument_RenderWithDpi_InvalidDpi_ThrowsArgumentOutOfRangeException
               - PdfDocument_RenderWithDpi_OutOfRangePageIndex_ThrowsArgumentOutOfRangeException
               - PdfDocument_RenderWithDpi_AfterDispose_ThrowsObjectDisposedException
+              - PdfDocument_RenderWithDpi_PropagatesOptionsBackgroundColor
 
           - id: CanvasNetPdf-PdfDocument-ContentStreamDispatch
             title: >-

docs/sysml2/model/canvas-net-pdf/pdf-document.sysml

@@ -2,7 +2,7 @@ package CanvasNet {
     part def PdfDocument {
         doc /* PDF document parser and page-rendering codec: parses cross-references, trailer, and page tree, reports page count/size/rotation, and interprets each page's content stream (path construction/painting, graphics-state stack, device color operators, generalized stream filter pipeline, image XObjects, and (Phase 4) simple TrueType font resolution/encoding and text-object/positioning/showing operators, with (Phase 6) automatic system/bundled-Liberation font-fallback substitution for a non-embedded font) to render real path geometry, images, and glyph outlines onto the returned Surface in real device color. */
 
-        comment sourceRef /* Source: src/DemaConsulting.CanvasNet.Pdf/PdfDocument.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Tokenizer.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.ObjectModel.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Xref.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.PageTree.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.ContentStream.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.GraphicsState.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.PathOps.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Color.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.Lzw.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.Ascii.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.RunLength.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Images.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.ToUnicode.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type0.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type1.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type3.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.FontFallback.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Text.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.CcittFax.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Encryption.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Functions.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.Shading.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.Tiling.cs, src/DemaConsulting.CanvasNet.Pdf/PdfPageInfo.cs, src/DemaConsulting.CanvasNet.Pdf/NamespaceDoc.cs */
+        comment sourceRef /* Source: src/DemaConsulting.CanvasNet.Pdf/PdfDocument.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Tokenizer.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.ObjectModel.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Xref.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.PageTree.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.ContentStream.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.GraphicsState.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.PathOps.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Color.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.Lzw.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.Ascii.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Filters.RunLength.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Images.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.ToUnicode.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type0.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type1.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Fonts.Type3.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.FontFallback.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Text.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.CcittFax.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Encryption.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Functions.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.Shading.cs, src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Patterns.Tiling.cs, src/DemaConsulting.CanvasNet.Pdf/PdfPageInfo.cs, src/DemaConsulting.CanvasNet.Pdf/PdfRenderOptions.cs, src/DemaConsulting.CanvasNet.Pdf/NamespaceDoc.cs */
         comment testRef /* Test: test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentTests.cs, test/DemaConsulting.CanvasNet.Pdf.Tests/PdfSystemIntegrationTests.cs, test/DemaConsulting.CanvasNet.Pdf.Tests/PdfFixtureTests.cs, test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentEncryptionTests.cs, test/DemaConsulting.CanvasNet.Pdf.Tests/PdfDocumentSymbolicEncodingTests.cs */
         comment designRef /* Design: docs/design/canvas-net-pdf/pdf-document.md */
         comment verificationRef /* Verification: docs/verification/canvas-net-pdf/pdf-document.md */