some question about the position of 'optimizer.zero_grad()'
Author: languandongCreated Dec 1, 2021Updated Dec 21, 2023
I think the correct way the code the training is that
optimizer.zero_grad()
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward and optimize
loss.backward()
optimizer.step()not that
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()Source: yunjey/pytorch-tutorial