BUG: stop vLLM inference when requests are aborted or disconnected
Author: amumu96Created Sep 17, 2026Updated Sep 17, 2026
Labelsbuggpu
Problem
Streaming vLLM requests can continue consuming GPU resources after the client disconnects. The current async-generator aclose() propagation is not deterministic across the REST, Xoscar, and vLLM boundaries. In addition, OpenAI-compatible request_id values are left inside generation config instead of being propagated to the model actor, so the existing abort endpoint returns NO_OP for vLLM.
Proposed change
- Propagate one request ID from the REST API through the model actor into vLLM.
- Track active vLLM request IDs and call the underlying engine abort API.
- Explicitly abort SSE requests when the client disconnects.
- Keep normal completion abort-free and preserve
DONE,NO_OP, andNOT_FOUNDoutcomes. - Cover explicit abort, disconnect cleanup, normal completion, unsupported engines, duplicate aborts, and concurrent requests.
Validation expectations
- Focused unit tests for REST request-ID propagation and vLLM lifecycle cleanup.
- Real GPU validation showing inference stops after disconnect and explicit abort.
Source: xorbitsai/inference