Wrong doc and function signature for 8-bit optim

Author: gau-nernstCreated Jun 30, 2024Updated Aug 28, 2026
LabelsDocumentationContributions WelcomeOptimizers

System Info

NA

Reproduction

Argument optim_bits and amsgrad is not used for 8-bit Adam

https://github.com/TimDettmers/bitsandbytes/blob/dada530149212d64d4b69534716202659ef37ec8/bitsandbytes/optim/adam.py#L77-L95

The document for optim_bits is also wrong in this case, because we are using 8 bits here.

https://github.com/TimDettmers/bitsandbytes/blob/dada530149212d64d4b69534716202659ef37ec8/bitsandbytes/optim/adam.py#L107-L110

I'm thinking of 2 ways to solve this:

  • Remove optim_bits and amsgrad arguments (actually even 32-bit Adam also doesn't use amsgrad argument)
  • If we want to keep the function signature the same, there should be a check that optim_bits == 8 and amsgrad == False

Expected behavior

NA

Source: bitsandbytes-foundation/bitsandbytes