Apple Silicon (MPS): create_model_from_path can segfault while loading bf16 checkpoints

Reproduction

On Apple Silicon, trainers loading a bf16 checkpoint from a model ID with the default initialization arguments can segfault while loading weights.

from datasets import Dataset
from trl import GRPOConfig, GRPOTrainer

trainer = GRPOTrainer(
    model="trl-internal-testing/tiny-Qwen3ForCausalLM",
    reward_funcs=lambda completions, **kwargs: [0.0] * len(completions),
    args=GRPOConfig(output_dir="out", per_device_train_batch_size=2, num_generations=2, report_to="none"),
    train_dataset=Dataset.from_dict({"prompt": ["hi", "hello"]}),
)

$ python -X faulthandler repro.py
Fatal Python error: Segmentation fault

This exited with code 139 in 4 of 4 fresh processes. The same happens with Qwen/Qwen2.5-0.5B-Instruct. It reduces to TRL's loader alone (3 of 3 runs):

from trl.trainer.utils import create_model_from_path

create_model_from_path("trl-internal-testing/tiny-Qwen3ForCausalLM")

Cause

create_model_from_path defaults device_map to "auto" unless the device is CPU, and defaults dtype to float32:

dtype = kwargs.get("dtype", "float32")
...
kwargs["device_map"] = None if PartialState().device.type == "cpu" else "auto"

On a Mac, PartialState().device.type is mps. A bf16 checkpoint loaded with these defaults therefore meets the conditions in huggingface/transformers#48029: the async loader can crash while converting bf16 weights to float32 and loading them onto MPS.

Not every load crashes. In my tests, device_map=None, disabling async loading, or matching dtype to the checkpoint each avoided it. Based on #48029, a float32 checkpoint should not trigger it either. create_model_from_path is used by SFT, DPO, KTO, RLOO, GRPO, Reward, Distillation, and several experimental trainers, so the default is not specific to GRPO.

Workarounds

All three of these allowed GRPOTrainer to construct with the 0.5B model:

• Set model_init_kwargs={"device_map": None} in the trainer config.
• Set model_init_kwargs={"dtype": "auto"} where MPS supports the checkpoint's dtype. I verified loading and trainer construction, not training with this setting.
• Set HF_DEACTIVATE_ASYNC_LOAD=1 before starting Python.

Preloading the policy model only partly avoids the problem. With full fine-tuning, no PEFT, and beta > 0, GRPO still creates its reference model through create_model_from_path and crashes. model_init_kwargs={"device_map": None} also avoids this reference-model crash, although GRPO first warns that those kwargs will be ignored because the policy is preloaded; the reference-model path subsequently uses them.

Faulthandler output

The process exits via SIGSEGV, so there is no ordinary Python exception traceback. In 3 of 4 runs, faulthandler produced multiple interleaved Fatal Python error headers, consistent with loader workers faulting concurrently. The readable stacks were:

Current thread (most recent call first):
  File "<site-packages>/transformers/core_model_loading.py", line 1240 in _materialize_copy
  File "<site-packages>/transformers/core_model_loading.py", line 1262 in _job
  File "<python3.12>/concurrent/futures/thread.py", line 59 in run
  File "<python3.12>/concurrent/futures/thread.py", line 93 in _worker

Main thread (most recent call first):
  File "<python3.12>/concurrent/futures/_base.py", line 451 in result
  File "<site-packages>/transformers/core_model_loading.py", line 970 in materialize_tensors
  File "<site-packages>/transformers/core_model_loading.py", line 1008 in convert
  File "<site-packages>/transformers/core_model_loading.py", line 1754 in convert_and_load_state_dict_in_model
  File "<site-packages>/transformers/modeling_utils.py", line 4456 in _load_pretrained_model
  File "<site-packages>/transformers/modeling_utils.py", line 4313 in from_pretrained
  File "<site-packages>/trl/trainer/utils.py", line 1203 in create_model_from_path
  File "<site-packages>/trl/trainer/grpo_trainer.py", line 343 in __init__
  File "repro.py", line 4 in <module>

Possible fix

For ordinary single-device MPS training, defaulting to device_map=None avoids the crashing Transformers path and leaves placement to Accelerate or the Trainer:

     if "device_map" not in kwargs:
-        kwargs["device_map"] = None if PartialState().device.type == "cpu" else "auto"
+        # device_map="auto" on MPS can crash Transformers' async loader (huggingface/transformers#48029)
+        kwargs["device_map"] = None if PartialState().device.type in ("cpu", "mps") else "auto"

The trade-off is that MPS models would load on CPU first, and automatic CPU or disk offload for models too large for accelerator memory would be lost. #4509 introduced the "auto" default to avoid the RAM-to-VRAM move, based on CUDA measurements. I have not measured the load-time difference on MPS. If huggingface/transformers#48029 is fixed upstream, this guard could be removed.

I can open a PR with this guard if that approach is acceptable.

What I tested

TRL 1.13.0 and Transformers 5.17.0; each case ran in a fresh process.

These tests cover loading and trainer construction. I did not run training steps with the patch or test other trainers.

Related

• huggingface/transformers#48029: underlying loader crash
• #4509: introduced the device_map="auto" default
• #6295: added the CPU exception to that default

System Info

- Platform: macOS-26.6.2-arm64-arm-64bit
- Python version: 3.12.12
- TRL version: 1.13.0
- PyTorch version: 2.14.0
- accelerator(s): MPS
- Transformers version: 5.17.0
- Accelerate version: 1.15.0
- Accelerate config: not found
- Datasets version: 5.0.1
- HF Hub version: 1.31.0
- bitsandbytes version: not installed
- DeepSpeed version: not installed
- Liger-Kernel version: not installed
- PEFT version: not installed
- vLLM version: not installed

Additional information:

• Hardware: Apple M4, 32 GB unified memory
• safetensors version: 0.8.0

Checklist

• I have checked that my issue isn't already filed (see open issues)
• I have included my system information
• Any code provided is minimal, complete, and reproducible (more on MREs)
• Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
• Any traceback provided is complete