巨型的内核 :)
克隆此仓库并运行:
git submodule update --init --recursive
pip install uv
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
uv pip install -e .首先,要编译 megakernel,请运行:
# from the repo root
export THUNDERKITTENS_ROOT=$(pwd)/ThunderKittens
export MEGAKERNELS_ROOT=$(pwd)
export PYTHON_VERSION=3.12 # adjust if yours is different
export GPU=H100 # options are {H100, B200}, else defaults to B200
cd demos/low-latency-llama
make
要与模型启动交互式聊天会话,请运行:
# from the repo root
python megakernels/scripts/llama_repl.py
要对 megakernel 进行基准测试,请运行:
# from the repo root
python megakernels/scripts/generate.py mode=mk prompt="tell me a funny joke about cookies" ntok=100
暂无开放 Issues，或尚未同步最近议题。