clean_contents(sanitize=True) drops glyph positioning, duplicating an overprinted leading glyph (regression in 1.28.0 from 1.27.2.3)
Description of the bug
Page.apply_redactions() corrupts unredacted text: the leading glyph of a
text span is duplicated when the page content stream is regenerated. It
reproduces even when the redaction removes nothing — a dummy annotation over an
empty area with every removal flag set to *_NONE — so it is the content-stream
regeneration itself, not the removal, that introduces the extra glyph.
The trigger is a span drawn with an "overprint" construct: a glyph is shown, then
0 0 TD moves back to the line-matrix origin, then a word is shown starting at
that same x. The two draws overlap exactly, so the page correctly reads faeces.
After apply_redactions() regenerates the stream, the draws no longer coincide
and both become visible, yielding ffaeces.
The duplication is not cosmetic: after redaction the two f glyphs render at
distinct x-origins (confirmed via get_text("rawdict")), so both the rendered
page and the extracted text show the extra letter.
This is a regression — correct in every release from 1.24.14 through 1.27.2.3, broken only in 1.28.0 (MuPDF 1.29.0):
| PyMuPDF | MuPDF | text after no-op apply_redactions() |
|---|---|---|
| 1.24.14 | 1.24.11 | faeces (correct) |
| 1.26.7 | 1.26.12 | faeces (correct) |
| 1.27.1 | 1.27.1 | faeces (correct) |
| 1.27.2 | 1.27.2 | faeces (correct) |
| 1.27.2.3 | 1.27.2 | faeces (correct) |
| 1.28.0 | 1.29.0 | ffaeces (leading glyph duplicated) |
Environment: PyMuPDF 1.28.0 / MuPDF 1.29.0 (broken), last good 1.27.2.3 / MuPDF 1.27.2; Python 3.13.13; Linux x86-64. The construct is emitted by real-world producers (first hit on an InDesign-generated PDF, where the extra glyph landed inside a math formula). Looks related to the MuPDF 1.29.0 text/font content-stream changes also behind the other 1.28.0 regressions (#5049, #5052).
How to reproduce the bug
Fully self-contained. Step 1 builds a ~600-byte PDF with the standard library only (Helvetica base-14, no embedded fonts, no third-party content); step 2 needs PyMuPDF.
Step 1 — make_repro_pdf.py (writes overprint.pdf):
"""Write a minimal, content-free PDF that triggers the redaction glyph-duplication bug.
The page draws the letter 'f' twice using the "overprint" construct:
(f)Tj # draw a lone 'f' at the line-matrix origin
0 0 TD # TD moves relative to the LINE matrix, not the pen -> back to origin
(faeces)Tj # draw 'faeces' starting at the SAME x -> its 'f' overprints the lone 'f'
Both f's paint at the same position, so the page shows a single, correct
"faeces" and text extraction returns "faeces".
"""
import sys
CONTENT = b"BT\n/F1 24 Tf\n1 0 0 1 100 700 Tm\n(f)Tj\n0 0 TD\n(faeces)Tj\nET\n"
OBJECTS = [
b"<< /Type /Catalog /Pages 2 0 R >>",
b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 800] "
b"/Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>",
b"<< /Length %d >>\nstream\n%s\nendstream" % (len(CONTENT), CONTENT),
b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
]
def build() -> bytes:
pdf = b"%PDF-1.7\n"
offsets = []
for i, body in enumerate(OBJECTS, start=1):
offsets.append(len(pdf))
pdf += b"%d 0 obj\n%s\nendobj\n" % (i, body)
xref_pos = len(pdf)
pdf += b"xref\n0 %d\n" % (len(OBJECTS) + 1)
pdf += b"0000000000 65535 f \n"
for off in offsets:
pdf += b"%010d 00000 n \n" % off
pdf += b"trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n" % (len(OBJECTS) + 1, xref_pos)
return pdf
if __name__ == "__main__":
out = sys.argv[1] if len(sys.argv) > 1 else "overprint.pdf"
with open(out, "wb") as fh:
fh.write(build())
print(f"wrote {out}")Step 2 — repro.py (extract, no-op redact, extract again):
import pymupdf
doc = pymupdf.open("overprint.pdf")
page = doc[0]
before = page.get_text().strip()
# A redaction that targets nothing: empty corner, every removal flag NONE.
page.add_redact_annot(pymupdf.Rect(2, 2, 5, 5))
page.apply_redactions(
text=pymupdf.PDF_REDACT_TEXT_NONE,
images=pymupdf.PDF_REDACT_IMAGE_NONE,
graphics=pymupdf.PDF_REDACT_LINE_ART_NONE,
)
after = page.get_text().strip()
print(f"PyMuPDF {pymupdf.version[0]} / MuPDF {pymupdf.version[1]}")
print(f" before redaction: {before!r}")
print(f" after redaction: {after!r}")Run:
python make_repro_pdf.py overprint.pdf
python repro.pyObserved on 1.28.0 (expected: after stays 'faeces'):
PyMuPDF 1.28.0 / MuPDF 1.29.0
before redaction: 'faeces'
after redaction: 'ffaeces'PyMuPDF version
1.28.0
Operating system
Linux
Python version
3.13
Source: pymupdf/PyMuPDF