[bug] --skip_until_initial_step 使用時、--gradient_accumulation_steps > 1 で再開位置を過剰にスキップして学習が終了する
Author: hybskgks28275Created Jul 20, 2026Updated Jul 21, 2026
事象
--resume で学習 state を再開し、--skip_until_initial_step を指定すると、--gradient_accumulation_steps が 2 以上の場合に、再開済み step 数より大幅に多くの epoch がスキップされます。 その結果、内部の step 数が負値まで減少し、学習が開始されずにすぐ終了します。
再現条件
--resume で state を再開 --skip_until_initial_step を指定 --gradient_accumulation_steps=4 再開 state の current_step=500 1 epoch あたりの dataloader batch 数が 100 の例
この条件では、500 optimizer step は 2000 dataloader batch に相当するため、本来は 20 epoch をスキップする必要があります。 しかし実際には 80 epoch がスキップされ、step 数が負値まで進みます。 再開 state は正常に読み込まれますが、--skip_until_initial_step 実行後に epoch のスキップ数が過剰となります。 最終的に学習ループへ入らず終了します。
ログ
running training / 学習開始
num train images * repeats / 学習画像の数×繰り返し回数: 100
num validation images * repeats / 学習画像の数×繰り返し回数: 0
num reg images / 正則化画像の数: 0
num batches per epoch / 1epochのバッチ数: 100
num epochs / epoch数: 30
batch size per device / バッチサイズ: 1
gradient accumulation steps / 勾配を合計するステップ数 = 4
total optimization steps / 学習ステップ数: 750
2026-07-21 07:38:46 INFO skipping 500 steps / 500ステップをスキップします train_network.py:1461
INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1494
2026-07-21 07:38:47 INFO skipping epoch 1 because initial_step (multiplied) is 2000 train_network.py:1514
INFO skipping epoch 2 because initial_step (multiplied) is 1900 train_network.py:1514
INFO skipping epoch 3 because initial_step (multiplied) is 1800 train_network.py:1514
INFO skipping epoch 4 because initial_step (multiplied) is 1700 train_network.py:1514
INFO skipping epoch 5 because initial_step (multiplied) is 1600 train_network.py:1514
INFO skipping epoch 6 because initial_step (multiplied) is 1500 train_network.py:1514
INFO skipping epoch 7 because initial_step (multiplied) is 1400 train_network.py:1514
INFO skipping epoch 8 because initial_step (multiplied) is 1300 train_network.py:1514
INFO skipping epoch 9 because initial_step (multiplied) is 1200 train_network.py:1514
INFO skipping epoch 10 because initial_step (multiplied) is 1100 train_network.py:1514
INFO skipping epoch 11 because initial_step (multiplied) is 1000 train_network.py:1514
INFO skipping epoch 12 because initial_step (multiplied) is 900 train_network.py:1514
INFO skipping epoch 13 because initial_step (multiplied) is 800 train_network.py:1514
INFO skipping epoch 14 because initial_step (multiplied) is 700 train_network.py:1514
INFO skipping epoch 15 because initial_step (multiplied) is 600 train_network.py:1514
INFO skipping epoch 16 because initial_step (multiplied) is 500 train_network.py:1514
INFO skipping epoch 17 because initial_step (multiplied) is 400 train_network.py:1514
INFO skipping epoch 18 because initial_step (multiplied) is 300 train_network.py:1514
INFO skipping epoch 19 because initial_step (multiplied) is 200 train_network.py:1514
INFO skipping epoch 20 because initial_step (multiplied) is 100 train_network.py:1514
INFO skipping epoch 21 because initial_step (multiplied) is 0 train_network.py:1514
INFO skipping epoch 22 because initial_step (multiplied) is -100 train_network.py:1514
INFO skipping epoch 23 because initial_step (multiplied) is -200 train_network.py:1514
INFO skipping epoch 24 because initial_step (multiplied) is -300 train_network.py:1514
INFO skipping epoch 25 because initial_step (multiplied) is -400 train_network.py:1514
INFO skipping epoch 26 because initial_step (multiplied) is -500 train_network.py:1514
INFO skipping epoch 27 because initial_step (multiplied) is -600 train_network.py:1514
INFO skipping epoch 28 because initial_step (multiplied) is -700 train_network.py:1514
INFO skipping epoch 29 because initial_step (multiplied) is -800 train_network.py:1514
INFO skipping epoch 30 because initial_step (multiplied) is -900 train_network.py:1514
INFO skipping epoch 31 because initial_step (multiplied) is -1000 train_network.py:1514
INFO skipping epoch 32 because initial_step (multiplied) is -1100 train_network.py:1514
INFO skipping epoch 33 because initial_step (multiplied) is -1200 train_network.py:1514
INFO skipping epoch 34 because initial_step (multiplied) is -1300 train_network.py:1514
INFO skipping epoch 35 because initial_step (multiplied) is -1400 train_network.py:1514
INFO skipping epoch 36 because initial_step (multiplied) is -1500 train_network.py:1514
INFO skipping epoch 37 because initial_step (multiplied) is -1600 train_network.py:1514
INFO skipping epoch 38 because initial_step (multiplied) is -1700 train_network.py:1514
INFO skipping epoch 39 because initial_step (multiplied) is -1800 train_network.py:1514
INFO skipping epoch 40 because initial_step (multiplied) is -1900 train_network.py:1514
INFO skipping epoch 41 because initial_step (multiplied) is -2000 train_network.py:1514
INFO skipping epoch 42 because initial_step (multiplied) is -2100 train_network.py:1514
INFO skipping epoch 43 because initial_step (multiplied) is -2200 train_network.py:1514
INFO skipping epoch 44 because initial_step (multiplied) is -2300 train_network.py:1514
INFO skipping epoch 45 because initial_step (multiplied) is -2400 train_network.py:1514
INFO skipping epoch 46 because initial_step (multiplied) is -2500 train_network.py:1514
INFO skipping epoch 47 because initial_step (multiplied) is -2600 train_network.py:1514
INFO skipping epoch 48 because initial_step (multiplied) is -2700 train_network.py:1514
INFO skipping epoch 49 because initial_step (multiplied) is -2800 train_network.py:1514
INFO skipping epoch 50 because initial_step (multiplied) is -2900 train_network.py:1514
INFO skipping epoch 51 because initial_step (multiplied) is -3000 train_network.py:1514
INFO skipping epoch 52 because initial_step (multiplied) is -3100 train_network.py:1514
INFO skipping epoch 53 because initial_step (multiplied) is -3200 train_network.py:1514
INFO skipping epoch 54 because initial_step (multiplied) is -3300 train_network.py:1514
INFO skipping epoch 55 because initial_step (multiplied) is -3400 train_network.py:1514
INFO skipping epoch 56 because initial_step (multiplied) is -3500 train_network.py:1514
INFO skipping epoch 57 because initial_step (multiplied) is -3600 train_network.py:1514
INFO skipping epoch 58 because initial_step (multiplied) is -3700 train_network.py:1514
INFO skipping epoch 59 because initial_step (multiplied) is -3800 train_network.py:1514
INFO skipping epoch 60 because initial_step (multiplied) is -3900 train_network.py:1514
INFO skipping epoch 61 because initial_step (multiplied) is -4000 train_network.py:1514
INFO skipping epoch 62 because initial_step (multiplied) is -4100 train_network.py:1514
INFO skipping epoch 63 because initial_step (multiplied) is -4200 train_network.py:1514
INFO skipping epoch 64 because initial_step (multiplied) is -4300 train_network.py:1514
INFO skipping epoch 65 because initial_step (multiplied) is -4400 train_network.py:1514
INFO skipping epoch 66 because initial_step (multiplied) is -4500 train_network.py:1514
INFO skipping epoch 67 because initial_step (multiplied) is -4600 train_network.py:1514
INFO skipping epoch 68 because initial_step (multiplied) is -4700 train_network.py:1514
INFO skipping epoch 69 because initial_step (multiplied) is -4800 train_network.py:1514
INFO skipping epoch 70 because initial_step (multiplied) is -4900 train_network.py:1514
INFO skipping epoch 71 because initial_step (multiplied) is -5000 train_network.py:1514
INFO skipping epoch 72 because initial_step (multiplied) is -5100 train_network.py:1514
INFO skipping epoch 73 because initial_step (multiplied) is -5200 train_network.py:1514
INFO skipping epoch 74 because initial_step (multiplied) is -5300 train_network.py:1514
INFO skipping epoch 75 because initial_step (multiplied) is -5400 train_network.py:1514
INFO skipping epoch 76 because initial_step (multiplied) is -5500 train_network.py:1514
INFO skipping epoch 77 because initial_step (multiplied) is -5600 train_network.py:1514
INFO skipping epoch 78 because initial_step (multiplied) is -5700 train_network.py:1514
INFO skipping epoch 79 because initial_step (multiplied) is -5800 train_network.py:1514
INFO skipping epoch 80 because initial_step (multiplied) is -5900 train_network.py:1514
INFO unet dtype: torch.bfloat16, device: cuda:0 train_network.py:1519Source: kohya-ss/sd-scripts