Proposal: end-to-end DINOv2 + ImageFolder tutorial for Datalab image audits
Motivation
Cleanlab already has strong tutorials for auditing image datasets with Datalab, and a separate tutorial that extracts image embeddings with a pretrained timm model for outlier detection. A workflow that may still be useful for newcomers is an end-to-end example starting from a local, class-organized image directory:
dataset/
├── class_a/
├── class_b/
└── class_c/The goal would be to show how to go from a standard torchvision.datasets.ImageFolder dataset to a Datalab audit with minimal setup and without first training a deep classifier.
Proposed tutorial scope
I would be happy to contribute a documentation example that:
- Loads a local
ImageFolderdataset and preserves image paths/class labels. - Extracts batched image embeddings with a small pretrained DINOv2 model.
- Caches the embeddings and supports CPU/CUDA execution.
- Demonstrates a fast features-only audit:
lab.find_issues(features=features) - Optionally demonstrates a more complete labeled-data workflow by fitting a lightweight linear classifier with cross-validation to obtain out-of-sample
pred_probs, then calling:lab.find_issues(features=features, pred_probs=pred_probs) - Reviews examples flagged as potential label issues, outliers, and near duplicates, with nearest-neighbor visual evidence.
- Keeps DINOv2/PyTorch dependencies tutorial-only rather than adding them as required cleanlab dependencies.
- Uses a small reproducible dataset and fixed seeds so the example remains practical to run.
This would not introduce a new label-error detection method; the intended contribution is a lower-friction, reproducible workflow that connects a common local image-dataset format to existing Datalab functionality.
Questions for maintainers
Before implementing this, would this tutorial be useful and in scope for the project?
If so:
- Would you prefer it in the main documentation or in the separate
cleanlab/examplesrepository? - Is
timmpreferred for loading DINOv2, or would another supported loader be better? - Should the first version stay features-only, or include the cross-validated linear-probe
pred_probsworkflow for label-issue detection? - Is there an existing issue or planned tutorial this should build on?
Thanks — I am happy to implement the tutorial and follow the project's development/documentation conventions if the direction is useful.
Source: cleanlab/cleanlab