ktransformers

工具介绍

经历异质LLM推论/精细优化的灵活框架

核心特点

• •Aug 26, 2026: Added native support for GLM-5.3-flash, bringing 1M-token context and multimodal input to consumer GPUs. (Tutorial)
• •Aug 17, 2026: LoRA fine-tuning now supports compatible AVX512 x86 CPUs, including AMD servers, without requiring AMX. (v0.7.0 Release Notes)
• •Aug 16, 2026: DeepSeek-V4-Flash on a single Ascend NPU with CPU expert offload. (Tutorial)
• •Aug 5, 2026: Introduced native block-FP8 LoRA fine-tuning, loading FP8 routed-expert weights directly from the checkpoint without materializing a complete BF16 copy. (PR #2141)
• •July 23, 2026: Added end-to-end BF16 full-parameter fine-tuning for MoE models, including complete checkpoint saving. (PR #2094)
• •June 21, 2026: MiniMax-M3 Day0 Support! (Tutorial)
• •June 17, 2026: GLM-5.2 Day0 Support! (Tutorial)
• •May 6, 2026: KTransformers at GOSIM Paris 2026 — "Agentic AI on Edge" track. We'll present KT's inference performance on consumer hardware.
• •May 02, 2026: DeepSeek-V4-Flash Support! (Tutorial)
• •Apr 30, 2026: KTransformers v0.6.1 refreshes kt-kernel inference and SFT docs with separate Inference and SFT Quick Start entry points.