You build an AI chat application.
A user sends: "Explain how distributed systems work." Your server calls an LLM API and starts streaming the answer: Everything looks great.
Then the user closes the browser tab.
The response disappears.
But what about the LLM request?
Is it still running?
If your server doesn't explicitly propagate cancellation, the answer may be yes.
And that's not just a correctness problem.
For an AI application, it can become a cost problem.
A user can abandon a generation after 500 tokens, while your backend continues paying for the remaining thousands of tokens.
In this article, we'll build the cancellation path for a TypeScript LLM server: Along the way, we'll look at: Why a browser closing a tab doesn't automatically cancel your LLM request How and work How to combine client disconnects with server-side timeouts How cancellation interacts with streaming responses When you should not cancel a request Why this is fundamentally a server-side TypeScript problem The Problem: The User Has Left, But Your LLM Hasn't Consider the simplest possible server: This works.
The browser sends a request: The LLM starts generating tokens, and the server streams them back to the browser.
But now: The browser is gone.
What happens to the call to the LLM?
Nothing automatically tells your application to cancel it.
The server and the upstream LLM request are separate operations.
Your server needs to explicitly connect their lifecycles.
First: Understand the Two Connections There are actually two HTTP connections here.
The browser controls Connection #1.
Your server controls Connection #2.
When the browser closes the tab: The LLM API doesn't magically know that the browser disappeared.
Your server has to propagate the cancellation: This is where comes in.
AbortController: The Cancellation Primitive The Web Platform already gives us a standard cancellation mechanism: The important part is: The is passed into the operation.
When: is called, APIs that support the signal can terminate the operation.
Node.js supports throughout its asynchronous APIs, including streams and HTTP-related operations.
This gives us a useful mental model: The controller doesn't need to know what it's cancelling.
It simply broadcasts: "Stop." Any operation that received its signal can react accordingly.
Detecting a Client Disconnect Now we need to answer the first question: How does the server know that the browser has disconnected?
With Hono, the request exposes the underlying request signal: This signal is aborted when the client connection is terminated.
So we can connect it directly to the LLM request: Now the lifecycle looks like this: If the browser disconnects: This is the critical connection that many first versions of AI applications miss.
But Client Disconnect Isn't the Only Reason to Cancel There is another failure mode.
What if the LLM API simply takes too long?
You don't want a request hanging forever.
So we have two independent cancellation conditions: We want: Cancel if either condition occurs.
Modern JavaScript gives us exactly that: creates a signal that aborts when any of the supplied signals aborts.
It is available in Node.js 20+ and later versions.
So: Now one signal represents both conditions.
The Complete Version Putting it together: There are now two ways the request can terminate: This is much better than manually maintaining separate timers and disconnect handlers.
Why This Matters Even More for Streaming Cancellation becomes particularly important when you're streaming LLM output.
Without streaming: With streaming: The request may stay alive for tens of seconds or even minutes.
That creates a much larger cancellation window.
The architecture is essentially a stream pipeline: The important thing is that the stream is not a special "AI" mechanism.
It's just a stream-processing pipeline.
Node.js and the Web Streams API provide cancellation mechanisms through , and stream operations can be terminated when their signal is aborted.
This is why understanding server-side streams is so useful when building AI applications.
Chunks Are Not Messages There's another subtle problem with streaming.
Suppose the LLM sends: You might imagine that your HTTP client receives exactly those chunks.
It doesn't have to.
The network might give you: then: then: A TCP chunk is not necessarily an application-level message.
So your streaming pipeline needs to buffer incomplete data: A is a natural fit: The pattern is: This same pattern appears when processing large files, SSE responses, and LLM streaming responses.
The key abstraction is incremental processing, not AI.
What About Backpressure?
There's one more reason to treat this as a stream pipeline.
What if the producer is faster than the consumer?
If data were allowed to accumulate indefinitely, memory usage could grow.
Streams solve this with backpressure.
Conceptually: When the downstream consumer cannot keep up, the stream machinery can stop pushing data upstream until capacity becomes available.
This is one of the major reasons streams are preferable to accumulating the entire response in memory.
And it is the same reason the following two approaches are fundamentally different: versus: For AI applications, incremental processing is what makes token-by-token responses possible.
The Cost Problem Now return to our original question.
Suppose: The exact financial impact depends on the model, provider, request, caching, and billing model.
But the engineering principle is simple: If an operation no longer has a consumer, you should explicitly decide whether the operation should continue.
For an interactive chat response, continuing is usually wasteful.
The user isn't going to read tokens that have nowhere to go.
Cancellation gives you a way to release the work: The benefit isn't only token cost.
You also release: an HTTP connection stream buffers server-side resources concurrency capacity provider-side generation work, where the upstream API honors cancellation But Don't Cancel Everything Here's the important architectural distinction.
Client disconnect does not always mean "cancel the task." Consider four operations.
1.
Interactive chat Cancel it.
The result has no value if the user has abandoned it.
2.
File indexing Don't necessarily cancel it.
The indexing operation is part of a persistent workflow.
The user's browser is merely observing the operation.
3.
Database write Usually, you want the database operation to complete.
The database write is a business operation, not a streaming response.
4.
Long-running Agent Consider an Agent run: This might take several minutes.
Binding the entire Agent lifecycle to an HTTP connection is usually the wrong architecture.
Instead: The frontend can then subscribe to the task: Now closing the browser doesn't necessarily destroy the Agent run.
This distinction is important: Cancellation is a business decision, not merely a technical decision.
A Better Mental Model Instead of thinking: "The browser disconnected, so cancel everything." Think: "What is the lifecycle of this operation?" There are two fundamentally different types of work: This distinction becomes increasingly important as an AI application grows.
One More Problem: Errors Cancellation is not necessarily a normal application error.
For example: Compare that with: These represent different things.
A production server should distinguish: expected operational failures external service failures programmer errors intentional cancellation A useful error hierarchy might look like: Then a global error handler can convert known application failures into consistent API responses while unexpected programmer errors are logged separately.
This kind of centralized error handling is especially important on servers because an unhandled error can affect many users rather than just one browser tab.
The Final Architecture Putting everything together: There are several independent pieces here: Type safety Streaming Cancellation Error handling None of