在自定义（自己的）数据集上进行迁移学习或微调 insightface

我目前正在使用 InsightFace 进行迁移学习,使用来自 ArcFace Torch 部分的 glint360k_cosface_r100_fp16_0.1 模型。然而,我遇到了数据集过拟合或欠拟合的问题,我不确定我做错了什么。以下是我遇到的问题: 1. 我的数据集包含 127 个人,每个人只有一张从不同角度拍摄的图像:正面视图、左侧和右侧的 3/4 视图、上视图、下视图以及左侧和右侧的侧面视图。这导致了 889 张图像的总数据集。 2. 最初,我将数据按文件夹级别分为 80% 用于训练和 20% 用于验证,这意味着每个人有 5 张训练图像和 2 张验证图像。这导致了欠拟合,可能是因为每个人的训练数据不足。 3. 为了解决这个问题,我先进行了数据增强,然后应用相同的 80/20 分割。然而,这导致了过拟合,因为我怀疑模型通过记忆增强图像中的模式而不是进行泛化而"作弊"。以下是代表我的方法的伪代码: BEGIN # ---- SETUP ENVIRONMENT ---- SET CUDA and OpenCV paths SET PyTorch memory allocation config # ---- IMPORT LIBRARIES ---- IMPORT required libraries (Torch, NumPy, OpenCV, InsightFace, etc.) # ---- DEFINE FaceDataset CLASS ---- CLASS FaceDataset: INITIALIZE dataset directory, transformations, and cache IF cache exists: LOAD dataset from cache ELSE: INITIALIZE face detection model (InsightFace) SCAN dataset directory FOR each image folder: FOR each image: DETECT face IF face detected: CROP and RESIZE to (112,112) STORE in dataset SAVE dataset to cache FUNCTION _detect_face(image): READ image CONVERT to RGB DETECT faces using InsightFace IF face detected: CROP, RESIZE, RETURN face ELSE: RETURN None FUNCTION __getitem__(index): RETURN image and label FUNCTION __len__(): RETURN number of samples # ---- DEFINE FaceRecognitionModel CLASS ---- CLASS FaceRecognitionModel: INITIALIZE ResNet50 backbone FREEZE lower layers, fine-tune upper layers ADD fully connected classifier with dropout FUNCTION forward(input): PASS through backbone PASS through classifier head RETURN output # ---- DEFINE TRAINING FUNCTION ---- FUNCTION train_model(model, train_loader, val_loader, criterion, optimizer, scheduler, num_epochs): INITIALIZE metrics storage SET early stopping threshold FOR epoch in range(num_epochs): IF warm-up phase: ADJUST learning rate # ---- TRAIN PHASE ---- SET model to training mode FOR batch in train_loader: LOAD input images and labels COMPUTE predictions CALCULATE loss BACKPROPAGATE and update weights …