[Parsing] Preserve PDF visual revision markings: text color and highlighted text in JSON/Markdown output

Description

Summary

LiteParse already does a good job extracting spatial text and, in current main, the Rust TextItem model appears to carry useful style metadata such as fill_color, stroke_color, link, and strike. It also already detects strikethrough from thin vector strokes/rectangles and renders strike as ~~...~~ in Markdown.

For regulatory/amendment PDFs, visual markings such as colored text and highlighted/background-filled text are semantically important. For example, EASA/aviation rulemaking PDFs often use:

• red strikethrough text for deleted provisions;
• cyan/yellow highlights for inserted or changed text;
• colored text for references or amended fragments.

It would be useful if LiteParse could preserve these visual revision markings directly in structured output and optionally in Markdown.

Use case

When feeding regulation PDFs to an LLM/RAG pipeline, plain text extraction loses important meaning. For amendment documents, the model needs to know not only the text content, but also whether a phrase was:

• deleted/struck through;
• inserted/highlighted;
• rendered in a distinctive text color.

Example desired Markdown:

Normal text <mark data-color="#00ffff">highlighted inserted text</mark> normal text.

<span data-color="#ff0000">red amended text</span>

~~deleted text~~

Current behavior

Strikethrough appears to be handled already in the Rust core and Markdown renderer.

However:

• text color/highlight fill does not appear to be represented in Markdown output;
• filled highlight rectangles are available as graphics, but do not appear to be assigned back to overlapping text items;
• Node bindings appear to expose only a subset of Rust TextItem fields, so color/style metadata may not be available to JS callers.

Document

npa-markings-highlights.pdf

Expected Output

Proposed behavior

Add first-class visual style metadata to parsed text items and optional Markdown rendering.

Structured output

Expose fields such as:

interface TextItem {
  text: string;
  x: number;
  y: number;
  width: number;
  height: number;

  fillColor?: string;       // e.g. "#ff0000" or ARGB/RGBA equivalent
  strokeColor?: string;
  highlightColor?: string;  // filled rect/background overlapping text
  strike?: boolean;
}

Markdown output

Add an opt-in config flag, for example:

visualStyleMode?: "off" | "html";

When enabled:

<mark data-color="#00ffff">highlighted text</mark>
<span data-color="#ff0000">colored text</span>
~~struck text~~

Default could remain off to avoid breaking existing Markdown consumers.

Suggested implementation

1. Keep current strikethrough behavior.
2. In extract.rs, add a pass similar to assign_links / assign_strikethrough:
   • collect filled GraphicPrimitive::Rect items;
   • ignore dark page banners, very small line rectangles, and black/white fills;
   • assign highlight_color to text items whose bbox center or sufficient bbox area overlaps the filled rect.
3. Preserve native text colors:
   • fill_color and stroke_color already seem to exist on Rust TextItem; expose them consistently in JSON and bindings.
4. Extend the Markdown inline style model:
   • include fill_color and highlight_color in the span style;
   • wrap highlighted spans in <mark data-color="...">;
   • wrap colored text spans in <span data-color="...">;
   • continue using ~~...~~ for strike.
5. Add tests:
   • synthetic PDF with cyan rectangle behind text;
   • synthetic PDF with red text;
   • synthetic PDF with red strikethrough;
   • mixed line with normal + highlighted + colored spans;
   • ensure dark banner backgrounds are not treated as highlights.

Reliability notes

This should be reliable for digital PDFs where highlights are vector rectangles and text is real PDF text.

It will not fully solve scanned PDFs where highlighting/color exists only as raster pixels. That would require OCR plus image/color segmentation and should probably be a separate feature or documented limitation.

Why this matters

For legal, regulatory, aviation, and standards documents, color and strikethrough are not cosmetic. They carry amendment semantics. Preserving them directly in LiteParse would make local, fast PDF parsing more useful for compliance and document-review agents without requiring downstream users to combine LiteParse with a separate PDF graphics parser.

Actual Output

Markdown without the visual color and highlights

Command Used

lit parse document-pdf --format markdown

LiteParse Version

2.1.1

Operating System

macOS (Apple Silicon)

Additional Context

No response