Fails on tencent/Hy3-FP8
Author: bani6809Created Aug 29, 2026Updated Aug 29, 2026
is this model format too new to be supported by heretic?
https://huggingface.co/tencent/Hy3-FP8
█░█░█▀▀░█▀▄░█▀▀░▀█▀░█░█▀▀ v1.4.0
█▀█░█▀▀░█▀▄░█▀▀░░█░░█░█░░ https://heretic-project.org
▀░▀░▀▀▀░▀░▀░▀▀▀░░▀░░▀░▀▀▀ https://github.com/p-e-w/heretic
Detected 4 CUDA device(s) (379.88 GB total VRAM)
CUDA Version: 13.0
Driver Version: 580.142
* CUDA 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (94.97 GB)
* CUDA 1: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (94.97 GB)
* CUDA 2: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (94.97 GB)
* CUDA 3: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (94.97 GB)
Loading model tencent.Hy3-FP8...
* Trying dtype auto...
* Failed ('NoneType' object has no attribute 'get')
* Trying dtype float16...
* Failed ('NoneType' object has no attribute 'get')
* Trying dtype bfloat16...
* Failed ('NoneType' object has no attribute 'get')
* Trying dtype float32...
* Failed ('NoneType' object has no attribute 'get')
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/admin/ai/heretic-llm/bin/heretic:8 in <module> │
│ │
│ 5 from heretic.main import main │
│ 6 if __name__ == '__main__': │
│ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ /home/admin/ai/heretic-llm/lib/python3.12/site-packages/heretic/main.py:1307 in main │
│ │
│ 1304 │ install() │
│ 1305 │ │
│ 1306 │ try: │
│ ❱ 1307 │ │ run() │
│ 1308 │ except BaseException as error: │
│ 1309 │ │ # Transformers appears to handle KeyboardInterrupt (or BaseException) │
│ 1310 │ │ # internally in some places, which can re-raise a different error in the handler │
│ │
│ /home/admin/ai/heretic-llm/lib/python3.12/site-packages/heretic/main.py:375 in run │
│ │
│ 372 │ │ elif choice is None or choice == "": │
│ 373 │ │ │ return │
│ 374 │ │
│ ❱ 375 │ model = Model(settings) │
│ 376 │ print() │
│ 377 │ print_memory_usage() │
│ 378 │
│ │
│ /home/admin/ai/heretic-llm/lib/python3.12/site-packages/heretic/model.py:169 in __init__ │
│ │
│ 166 │ │ │ break │
│ 167 │ │ │
│ 168 │ │ if self.model is None: │
│ ❱ 169 │ │ │ raise Exception("Failed to load model with all configured dtypes.") │
│ 170 │ │ │
│ 171 │ │ self._apply_lora() │
│ 172 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
Exception: Failed to load model with all configured dtypes.Source: p-e-w/heretic