KappaTuneSelector does not dequantize bitsandbytes 8-bit weights, computing condition numbers on raw int8 codes

System Info

• peft: main (0e8d0ae8)
• transformers: 5.16.1
• torch: 2.13.0
• Python 3.11
• Affects bitsandbytes 8-bit models (load_in_8bit=True); independent of platform

Who can help?

@benjaminbossan @githubnemo

Reproduction

KappaTuneSelector._compute_kappas tries to dequantize bnb 8-bit weights, but it reads CB/SCB from the wrong object:

https://github.com/huggingface/peft/blob/0e8d0ae8ab94f189f28b845e293d7452d7892d91/src/peft/helpers.py#L446-L455

weight = module.weight
if bnb is not None:
    if hasattr(weight, "quant_state"):  # 4-bit
        w = bnb.functional.dequantize_4bit(weight.data, weight.quant_state).float()
    elif hasattr(weight, "state") and hasattr(weight.state, "CB"):  # int8
        w = bnb.functional.int8_vectorwise_dequant(weight.state.CB, weight.state.SCB).float()
    else:
        w = weight.data.detach().float()

In bitsandbytes, CB and SCB are attributes of the Int8Params weight itself (self.CB = CB, self.SCB = SCB in bitsandbytes/nn/modules.py). state (MatmulLtState) belongs to the Linear8bitLt module, not to its weight. So hasattr(weight, "state") is False for 8-bit weights, and the int8 branch is never taken.

The code then falls through to w = weight.data.detach().float(), which computes the SVD on the raw int8 codes without applying the per-row scales. Because each row is scaled by a different factor, the resulting condition numbers (and therefore the selected targets) differ from those of the actual weights. This happens silently, with no error or warning.

For comparison, PEFT's own dequantize_bnb_weight in src/peft/utils/integrations.py handles 8-bit correctly by taking the module's state and falling back to weight.SCB.

The existing GPU test test_kappatune_with_4bit_model in tests/test_gpu_examples.py only covers 4-bit, so this path isn't exercised.

Note: I found this by reading the PEFT and bitsandbytes source; I don't have a GPU available to run it. A reproducer along these lines should show the problem (untested):

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft.helpers import KappaTuneSelector

model_id = "facebook/opt-125m"

model_8bit = AutoModelForCausalLM.from_pretrained(
    model_id, quantization_config=BitsAndBytesConfig(load_in_8bit=True), device_map="cuda"
)
module = model_8bit.model.decoder.layers[0].self_attn.q_proj
print(type(module.weight).__name__, hasattr(module.weight, "state"), hasattr(module.weight, "CB"))
# expected: Int8Params False True  -> int8 branch is skipped

model_fp = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map="cuda")

kappa_8bit = KappaTuneSelector(model_8bit, show_progress=False)
kappa_fp = KappaTuneSelector(model_fp, show_progress=False)
kappa_8bit._compute_kappas()
kappa_fp._compute_kappas()

name = "model.decoder.layers.0.self_attn.q_proj"
print(kappa_8bit._condition_numbers[name], kappa_fp._condition_numbers[name])
# expected: values differ noticeably, since the 8-bit one is computed on unscaled int8 codes

Expected behavior

For 8-bit bnb models, KappaTune should dequantize the weights (e.g. via weight.CB/weight.SCB, or by reusing dequantize_bnb_weight(module.weight, state=module.state)) before computing condition numbers. Its target selection should then approximately match the selection for the same model loaded unquantized. An 8-bit case could be added next to test_kappatune_with_4bit_model.