Synthetic-space detection may insert spaces for small positioning differences

This is a separate issue from #4062.

The threshold adjustment affects synthetic whitespace insertion, while #4062 only fixes the text matrix reported to visitor_text().

Additional observation about synthetic-space detection

I also tested adding a small tolerance to the synthetic-space condition in crlf_space_check() in _text_extraction/__init__.py.

Using a threshold multiplier of 1.05 makes the extracted text consistent with the text extracted by Adobe Acrobat for the PDF files I tested:

• For the PDF from #2513, " T EL" becomes "T EL".
• For the PDF from #3219, "WO RD" becomes "WORD".

I tested the following change:

python复制

    LINE_BREAK_THRESHOLD = 0.8
    SPACE_INSERTION_THRESHOLD = 1.05

    # ...

    try:
        if abs(moved_height) > LINE_BREAK_THRESHOLD * min(str_height * scale_prev_y, font_size * scale_y):
            if (output + text)[-1] != "\n":
                output += text + "\n"
                if visitor_text is not None:
                    visitor_text(
                        text + "\n",
                        memo_cm,
                        memo_tm,
                        font_resource,
                        font_size,
                    )
                text = ""
        elif (
            (moved_width >= SPACE_INSERTION_THRESHOLD * (spacewidth + str_widths) * scale_prev_x)
            and (output + text)[-1] != " "
        ):
            print()
            print("[SPACE INSERTED]")
            print("text before     =", repr(text))
            print("moved_width     =", moved_width)
            print("spacewidth      =", spacewidth)
            print("str_widths      =", str_widths)
            print("tm_prev         =", tm_prev)
            print("tm_matrix       =", tm_matrix)
            print("memo_tm         =", memo_tm)

            text += " "

            print("text after      =", repr(text))

    except Exception:
        pass

This also changes the expected result of the following existing test:

复制

tests/test_base.py::test_text_string_object__wrongly_detected_bom

The difference is that several synthetic spaces are no longer inserted. For example:

复制

Before: 参 z慧
After:  参z慧

Before: 调 uC
After:  调uC

The complete output after applying the threshold is also consistent with the text extracted by Adobe Acrobat.

This test appears to primarily cover BOM detection, while its current expected value also fixes the output of the synthetic-space heuristic. The failure may therefore indicate that the expected extracted text should be reviewed if the spacing threshold is adjusted, rather than necessarily indicating a regression in BOM handling.

However, the value 1.05 is currently based on the tested PDF files and has not been comprehensively evaluated across different fonts, text matrices, writing directions, and text-positioning patterns. I therefore consider this an additional observation rather than a fully validated fix.

Originally posted by @ssjkamei in #2932