#7202·trl

Audit the trl-internal-testing repos that have no generation script

Author: albertvillanovaCreated Sep 14, 2026Updated Sep 17, 2026

Follow-up to #7186, where we agreed on the principle: every repo under trl-internal-testing should have its generation script in this repository, datasets included. A repo nobody can regenerate is a repo nobody can reason about.

Deleting a Hub repo is irreversible, and the set is not uniform, so this issue goes through them one by one. Nothing gets deleted until it is agreed here.

What was checked

Every repo in the org cross-referenced against the generation scripts in scripts/generate_tiny_models/ and scripts/, then every deletion candidate cross-referenced against every repo in the huggingface org that mentions trl-internal-testing: this repository, trl-internal, transformers, peft, diffusers, sentence-transformers, blog, h4, community-evals, optimum-habana, open-r1 and the new-model-addition-* repos.

75 model repos, 61 generation scripts. The grouping below follows @qgallouedec's review.

Transfer to the consumer's test org

Agreed approach: rather than writing scripts for repos whose consumers are elsewhere, move the repo to the consumer's own test org. A Hub transfer redirects the old id and preserves download counts, so their CI keeps passing and they can update ids at their own pace. Each needs the receiving team's agreement.

  • tiny-random-LlamaForCausalLM: peft, 27 references, to peft-internal-testing
  • tiny-Llama4ForCausalLM: peft, 12 references, to peft-internal-testing
  • tiny-DbrxForCausalLM: transformers, also sentence-transformers, to hf-internal-testing
  • tiny-BloomForCausalLM: proposed addition to this group. transformers uses it at tests/models/auto/test_tokenization_auto.py:544 as a fixture for a repo whose tokenizer_class no longer exists, so it cannot be regenerated without destroying what it tests. Our own 9 references are all for "a tokenizer with no chat template", which tiny-GPT2LMHeadModel also provides and does have a script.
  • example-images: proposed addition. A documentation image host with nothing to generate, used by transformers docs.
  • tiny-Qwen3_5ForConditionalGeneration: the case this approach does not cover. Its only consumer is trl-internal, 17 references, which has no separate test org.

Explicit exception to the rule

  • dummy-GPT2-correct-vocab: agreed to leave the generation script on the model card. It predates the tiny- convention, and push_to_hub in _common.py builds ids as {prefix}-{ClassName}[-{suffix}], which can never produce this name.

No consumer anywhere, clear to delete

Deletion blocked by a consumer outside this repository

The sweep found these after the grouping above was first written, so treat any earlier agreement on them as superseded.

sentence-transformers is a systematic consumer rather than incidental: its update_transformers_tiny_models.py scans trl-internal-testing as one of its source orgs, and the resulting map drives parametrized tests through a conftest.

A caveat on download counts

They are tempting as a usage signal and they are unreliable in both directions. tiny-PeftModel is loaded by three tests on every run and reports zero downloads, because the counter keys on config.json and that repo only has adapter_config.json. In the other direction, several repos with no identifiable consumer anywhere carry six-figure monthly counts. So the grouping above is based on reading code, not on traffic.

Proposed rule

A repo stays if it has a generation script here, a named consumer elsewhere, or a recorded reason it can have neither. Everything else is a deletion candidate. Ticking a box means we agreed on that repo.