How to include page numbers in markdown / Conditional OCR when text layer is missing
Hi, thank you for the great project! I've been reading the README and documentation carefully, but I couldn't find answers to the following two questions. I would appreciate any clarification you can provide:
1. Including Page Numbers in Markdown Output
Is there a way to include page numbers in the output when using export_to_markdown() or save_as_markdown()?
I tried using page_break_placeholder="<!-- page {page_no} -->", but it seems that the {page_no} variable is not recognized and is inserted as a literal string. Is there a built-in way to insert the actual page number into the markdown output?
2. Conditional OCR Based on Text Layer
I'm trying to build a pipeline that:
- uses the text layer when it exists, and
- falls back to OCR only when the text layer is missing.
However, when I set do_ocr = True, it doesn't seem to run OCR properly unless I also set force_full_page_ocr = True. But with force_full_page_ocr = True, even PDFs that already have a text layer get OCRed, which is not ideal.
I confirmed using fitz (PyMuPDF) that the document does have an existing text layer. Still, docling either doesn't pick it up or forces OCR regardless.
Is there a recommended or correct way in Docling to conditionally perform OCR only if the PDF has no usable text layer?
This is my code, and I hope I can solve this problem.
def conver_pdf_to_assets(file_path, output_folder):
logging.basicConfig(level=logging.INFO)
render_options = PdfPipelineOptions()
render_options.images_scale = IMAGE_ZOOM_FACTOR
render_options.generate_picture_images = True
render_options.generate_page_images = True
render_options.do_ocr = True
render_options.do_table_structure = True
render_options.table_structure_options.do_cell_matching = True
ocr_options = TesseractCliOcrOptions(force_full_page_ocr=True)
render_options.ocr_options = ocr_options
pdf_processor = DocumentConverter(
format_options= {
InputFormat.PDF: PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
accelerator_options=AcceleratorOptions(device=AcceleratorDevice.CUDA, num_threads=8),
pipeline_options= render_options,
),
})
conversion_result = pdf_processor.convert(file_path)
table_index = 0
figure_index=0
for content_element, _ in conversion_result.document.iterate_items():
if isinstance(content_element, TableItem):
table_index += 1
img = content_element.get_image(conversion_result.document)
if img :
table_img_file = output_folder/f"table-{table_index}.png"
with table_img_file.open('wb') as tbl_out:
content_element.get_image(conversion_result.document).save(tbl_out, format='PNG')
else:
logging.warning(f"Table {table_index} does not have an image associated with it.")
if isinstance(content_element, PictureItem):
figure_index += 1
figure_img_file = output_folder/f"figure-{figure_index}.png"
with figure_img_file.open('wb') as fig_out:
content_element.get_image(conversion_result.document).save(fig_out, format='PNG')
embedded_md_path = output_folder/f"AJ-with-images.md"
conversion_result.document.save_as_markdown(
filename=embedded_md_path,
image_mode=ImageRefMode.REFERENCED,
page_break_placeholder=f"<!-- page {page_no} -->"
)
Thank you in advance for your help — and again, thank you for building and sharing this powerful tool!
Source: docling-project/docling