[Bug] NewCM text fonts are misclassified as formulas, leaving body text untranslated

Before you submit

• I have searched existing issues
• I spent at least 5 minutes investigating and preparing this report
• I confirmed this is not caused by a network issue
• I have fully read and understood the README
• I am certain that this issue is with BabelDOC itself and can be reproduced independently of the translation service
• I have linked the original PDF below
• I have included the relevant diagnostic output below
• I confirmed that the issue is still present in the latest release (v0.6.4) and current main branch
• I am aware that the issue section is only for clearly reproducible BabelDOC core bugs

Environment

- OS: macOS 26.5.1 (Build 25F80)
- Python: 3.12.13
- Local BabelDOC: 0.6.2 (bundled with pdf2zh-next)
- Latest BabelDOC version checked: 0.6.4
- pdf2zh-next: 2.9.0

Describe the bug

BabelDOC incorrectly classifies regular New Computer Modern text fonts as formula fonts.

The affected PDF uses these fonts:

• NewCM10-Regular for most body text
• NewCM10-Bold for headings and bold text
• NewCM10-Italic for italic text
• NewCMMath-Book for actual mathematical content

However, is_formulas_font() returns True for all of them:

NewCM10-Regular: True
NewCM10-Bold: True
NewCM10-Italic: True
NewCMMath-Book: True

As a result, almost all normal body text is converted into formula placeholders such as {v1}, {v2}, and {v3} before it reaches the translation engine.

The translation engine therefore receives inputs similar to:

[
  {
    "id": 0,
    "input": "{v1},{v2}{v3}",
    "layout_label": "plain text"
  },
  {
    "id": 1,
    "input": "{v1}{v2}{v3}{v4},{v5}",
    "layout_label": "plain text"
  }
]

Because the translation prompt correctly instructs the model to preserve formula placeholders, the model returns them unchanged. BabelDOC then reports:

Translation result is the same as input, fallback.
Fallback to simple translation.

The fallback path receives the same placeholders, so changing the translation model, JSON mode, or prompt cannot recover the original text.

Steps to Reproduce

1. Download the original PDF linked below.
2. Translate it with BabelDOC or pdf2zh-next using any supported LLM translator.
3. Inspect the generated PDF: most body text remains in English, while only some punctuation or text using other fonts may change.
4. Run this minimal reproduction:

from babeldoc.format.pdf.document_il.utils.formular_helper import (
    is_formulas_font,
)

font_names = [
    "NewCM10-Regular",
    "NewCM10-Bold",
    "NewCM10-Italic",
    "NewCMMath-Book",
]

for font_name in font_names:
    print(font_name, is_formulas_font(font_name, None))

Actual output:

NewCM10-Regular True
NewCM10-Bold True
NewCM10-Italic True
NewCMMath-Book True

A custom formula pattern does not prevent the misclassification either:

is_formulas_font("NewCM10-Regular", r".*Math.*")

Actual result:

True

Expected Behavior

Regular New Computer Modern text fonts should not be treated as formula fonts:

NewCM10-Regular: False
NewCM10-Bold: False
NewCM10-Italic: False
NewCMMath-Book: True

Normal English body text should remain available to the translation engine, while only actual mathematical content should be replaced with formula placeholders.

Relevant Log Output or Screenshots

Relevant LLM input:

[
  {"id": 0, "input": "{v1},{v2}{v3}", "layout_label": "plain text"},
  {"id": 1, "input": "{v1}{v2}{v3}{v4},{v5}", "layout_label": "plain text"}
]

Relevant BabelDOC warnings:

Translation result is the same as input, fallback.
Fallback to simple translation.

Font usage in the reproduction PDF:

NewCM10-Regular: 97689 characters
NewCM10-Bold: 6105 characters
NewCM10-Italic: 884 characters
NewCMMath-Book: 2674 characters

This explains why almost the entire document is affected.

Original PDF File

Stanford CS336 Assignment 1: Basics

Suspected Root Cause

The precise formula-font pattern in:

babeldoc/format/pdf/document_il/utils/formular_helper.py

contains:

r"|.*NewCM.*"
r"|.*NewComputerModern.*"

These expressions match both mathematical fonts and ordinary text fonts such as NewCM10-Regular.

Additionally, the precise built-in pattern is evaluated before the user-provided formular_font_pattern:

if re.match(precise_formula_font_pattern, font):
    return True
elif re.match(pattern_text, font):
    return False
elif re.match(broad_formula_font_pattern, font):
    return True

Therefore, --formular-font-pattern cannot be used as a workaround for this case.

The same broad NewCM expressions are still present on the current main branch:

https://github.com/funstory-ai/BabelDOC/blob/main/babeldoc/format/pdf/document_il/utils/formular_helper.py

Suggested Fix

A minimal fix may be to narrow the built-in New Computer Modern expressions to actual math font names:

r"|.*NewCMMath.*"
r"|.*NewComputerModernMath.*"

Suggested regression tests:

assert not is_formulas_font("NewCM10-Regular", None)
assert not is_formulas_font("NewCM10-Bold", None)
assert not is_formulas_font("NewCM10-Italic", None)
assert is_formulas_font("NewCMMath-Book", None)
assert is_formulas_font("NewCMMath-Bold", None)
assert not is_formulas_font("ABCDEF+NewCM10-Regular", None)

I would be happy to submit a small PR with the narrowed patterns and regression tests if the maintainers agree with this approach.

Additional Context

The translation API itself works correctly when it receives normal English text. The failure happens earlier in BabelDOC's Parse Formulas and Styles stage, before the translation request is created.

Related: #280 proposes better custom formula-font whitelist and blacklist support, but it does not cover this specific built-in false positive.