v2.0.0 test suite cannot run because datalab-to/pdfs is unavailable
Restore or provide access to the PDF fixtures required by the test suite
Problem
The Marker test suite depends on PDF fixtures from the Hugging Face dataset:
https://huggingface.co/datasets/datalab-to/pdfs
However, the dataset is currently unavailable. Loading it from the v2.0.0
test suite fails with:
datasets.exceptions.DatasetNotFoundError:
Dataset 'datalab-to/pdfs' doesn't exist on the Hub or cannot be accessed.The dataset page/API currently returns HTTP 404 (Repository not found),
including when authenticated with a valid Hugging Face account.
As a result, most of the Marker test suite cannot run.
Affected Tests
The following tests load PDFs from this dataset through:
datasets.load_dataset("datalab-to/pdfs", split="train")The test fixture is defined in:
tests/conftest.pyThe required PDF files are:
A17_FlightPlan.pdf
adversarial.pdf
adversarial_rot.pdf
arxiv_test.pdf
bio_pdf.pdf
form_1040.pdf
handwritten.pdf
hindi_judgement.pdf
multicol-blocks.pdf
population_stats.pdf
pres.pdf
thinkpython.pdf
water_damage.pdfThese are not interchangeable files. Several tests assert specific document content and layout, for example:
thinkpython.pdfmust contain theThink Pythontitle and expected font metadata.adversarial.pdfmust containSubspace Adversarial Training.water_damage.pdfmust contain a table and the textваріант.multicol-blocks.pdfmust containCascading, and the Auxiliary Problem Principle.pres.pdfmust contain1.2E-38.table_ex.pdfmust contain theParticipantstable.population_stats.pdfmust contain the expected footnote structure.
Renaming unrelated PDFs would make the tests pass falsely and would not be a valid workaround.
Reproduction
Using Marker v2.0.0:
uv sync --all-extras --all-groups
pytest -raThe test collection succeeds, but fixtures that depend on datalab-to/pdfs
fail during setup because the dataset cannot be accessed.
Expected Behavior
The test suite should have a stable and reproducible source for all required fixtures.
Requested Resolution
Could the maintainers please do one of the following?
- Restore the
datalab-to/pdfsdataset at its current URL. - Provide the new dataset URL if it has been moved or renamed.
- Make the dataset accessible to the public test workflow.
- Provide a downloadable archive or repository containing the required fixture files.
- Update the tests to use a maintained fixture source.
- Document the required authentication and access procedure if the dataset is intentionally private.
If the dataset is intentionally private, the CI workflow should ideally use a versioned fixture artifact or another reproducible mechanism that does not require personal Hugging Face credentials.
Additional Note
The test suite also references non-PDF files from the same dataset:
lambda.pptx
manual.epub
china.html
gatsby.docx
single_sheet.xlsxThose files may need to be restored or relocated as well for the complete test suite to pass.
Source: datalab-to/marker