#1084·marker

v2.0.0 test suite cannot run because datalab-to/pdfs is unavailable

Author: scotthuang1989Created Aug 18, 2026Updated Aug 18, 2026

Restore or provide access to the PDF fixtures required by the test suite

Problem

The Marker test suite depends on PDF fixtures from the Hugging Face dataset:

https://huggingface.co/datasets/datalab-to/pdfs

However, the dataset is currently unavailable. Loading it from the v2.0.0 test suite fails with:

datasets.exceptions.DatasetNotFoundError:
Dataset 'datalab-to/pdfs' doesn't exist on the Hub or cannot be accessed.

The dataset page/API currently returns HTTP 404 (Repository not found), including when authenticated with a valid Hugging Face account.

As a result, most of the Marker test suite cannot run.

Affected Tests

The following tests load PDFs from this dataset through:

python
datasets.load_dataset("datalab-to/pdfs", split="train")

The test fixture is defined in:

tests/conftest.py

The required PDF files are:

A17_FlightPlan.pdf
adversarial.pdf
adversarial_rot.pdf
arxiv_test.pdf
bio_pdf.pdf
form_1040.pdf
handwritten.pdf
hindi_judgement.pdf
multicol-blocks.pdf
population_stats.pdf
pres.pdf
thinkpython.pdf
water_damage.pdf

These are not interchangeable files. Several tests assert specific document content and layout, for example:

  • thinkpython.pdf must contain the Think Python title and expected font metadata.
  • adversarial.pdf must contain Subspace Adversarial Training.
  • water_damage.pdf must contain a table and the text варіант.
  • multicol-blocks.pdf must contain Cascading, and the Auxiliary Problem Principle.
  • pres.pdf must contain 1.2E-38.
  • table_ex.pdf must contain the Participants table.
  • population_stats.pdf must contain the expected footnote structure.

Renaming unrelated PDFs would make the tests pass falsely and would not be a valid workaround.

Reproduction

Using Marker v2.0.0:

bash
uv sync --all-extras --all-groups
pytest -ra

The test collection succeeds, but fixtures that depend on datalab-to/pdfs fail during setup because the dataset cannot be accessed.

Expected Behavior

The test suite should have a stable and reproducible source for all required fixtures.

Requested Resolution

Could the maintainers please do one of the following?

  1. Restore the datalab-to/pdfs dataset at its current URL.
  2. Provide the new dataset URL if it has been moved or renamed.
  3. Make the dataset accessible to the public test workflow.
  4. Provide a downloadable archive or repository containing the required fixture files.
  5. Update the tests to use a maintained fixture source.
  6. Document the required authentication and access procedure if the dataset is intentionally private.

If the dataset is intentionally private, the CI workflow should ideally use a versioned fixture artifact or another reproducible mechanism that does not require personal Hugging Face credentials.

Additional Note

The test suite also references non-PDF files from the same dataset:

lambda.pptx
manual.epub
china.html
gatsby.docx
single_sheet.xlsx

Those files may need to be restored or relocated as well for the complete test suite to pass.