Request.pushErrorMessage stores null when Error.stack is missing, poisoning persisted requests
Which package is this bug report for?
@crawlee/core (observed through @crawlee/cheerio / CheerioCrawler)
Issue description
Request.pushErrorMessage(error) assumes that every Error has a string stack. When an error has stack === null or stack === undefined, it appends that value to request.errorMessages, although RequestOptions.errorMessages only accepts strings.
The invalid request can then fail while Crawlee reclaims, persists, marks handled, or reconstructs it:
ArgumentError: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null`We encountered this after an HTTP transport emitted a response-stream/TLS error with a valid message but no stack. The persistence failure prevented the request from settling normally and, in a long-lived shared crawler runtime, blocked later work that was waiting for that runtime to drain.
This appears to be the same underlying problem reported in #1351. That report was closed after maintainers requested a reproduction. The sample below provides a deterministic reproduction and shows where the invalid state originates.
The current v3.18.1 implementation still selects errorOrMessage.stack without verifying that it is a string:
https://github.com/apify/crawlee/blob/v3.18.1/packages/core/src/request.ts#L344-L373
Expected behavior: pushErrorMessage() should always append a string. For an Error without a stack, it could fall back to error.message, then String(error), or omit a non-string value.
Code sample
import { Request } from "@crawlee/cheerio";
const request = new Request({ url: "https://example.com" });
const error = new Error("response stream failed");
// Some transport/runtime errors can arrive without a stack.
Object.defineProperty(error, "stack", { value: null });
request.pushErrorMessage(error);
console.log(request.errorMessages); // [null]
// Reproduces what happens when the invalid request is reconstructed/persisted.
new Request({
url: request.url,
errorMessages: request.errorMessages,
});Result:
ArgumentError: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null`Package version
Reproduced with @crawlee/[email protected] through @crawlee/[email protected]. The same unchecked stack assignment is present in the latest v3.18.1 source linked above.
Node.js version
Production runtime: Bun 1.4.0. The reproduction does not rely on Bun-specific APIs; it only constructs a Crawlee Request and an Error whose stack is null.
Operating system
macOS arm64 for the minimal reproduction; Linux arm64 in the affected worker runtime.
I have tested this on the next release
Not tested.
Other context
Our downstream workaround uses Crawlee's public errorHandler hook to normalize non-string entries in request.errorMessages before request persistence. We are not patching Crawlee locally.
Source: apify/crawlee