nano-vllm · Issues· 96 open

[Bug] scheduler: `assert scheduled_seqs` crashes engine when decode runs out of KV-cache blocks

[Bug] nano-vllm cannot start on Windows

Potential TP correctness issue: BlockManager capacity may rely on per-rank local KV-cache estimation

Tensor parallel shared-memory RPC lacks worker acknowledgement and can corrupt commands

Low inference speed in A100 PCIE GPU

Lightweight Go load-balancing proxy for nano-vllm

[Bug] may_append allocates new block one token too late, causing KV cache write to unallocated block

Documentation Enhancement Suggestion

[change] Int8 KV Cache + Async Pipeline + Head-major reordering for 22% Throughput Boost

Introducing Quantized Inference to Nano-vLLM - A Lightweight FP8 Runtime for Qwen Models

[Feature Request] Support weightless RMSNorm (for FlashNorm weight folding trick)

[Discussion] As vLLM-Omni is also complex, I tried building a minimal version (~1k LOC) inspired by nano-vllm

[Discussion] Duplicate blocks for same-step prefix sharing

Implement support for Qwen3.5

[BUG] Crashes when the prompt length exactly equals kvcache_block_size