Screenshots always time out on pages with a cookie banner since 0.33 (autoconsent re-layout races the screenshot)

Describe the Bug

Since upgrading 0.32.0 → 0.33.2, page.screenshot() always hits CRAWLER_SCREENSHOT_TIMEOUT_SEC on any page that has a cookie/consent banner. Not a single screenshot has been stored since the upgrade (asset count frozen at 952, last one from before the upgrade).

The cause is autoconsent, not a slow machine and not the new Chrome image.

crawlPage does roughly this:

goto(url, { waitUntil: "domcontentloaded" })
waitForLoadState("networkidle", { timeout: 5000 })
await waitForPageLoadAndAutoconsent(pageLoad, autoconsentHandle, abortSignal)
capturePageAssets(...)   // content() and screenshot() in parallel

Autoconsent dismisses the consent dialog, which triggers a re-layout and unblocks content the banner was gating. The page then starts loading and repainting heavily — and that is exactly the moment capturePageAssets fires the screenshot. The compositor never reaches a stable frame, so the screenshot never completes.

The tell-tale sign that this is a hang and not slowness: the screenshot always consumes the entire timeout, never slightly more than it.

Raising the timeout does not help — it only lengthens the wait before the same failure.

Steps to Reproduce

1. Run 0.33.2 with default CRAWLER_STORE_SCREENSHOT=true and default CRAWLER_ENABLE_AUTOCONSENT (on).
2. Bookmark any URL that shows a cookie/consent banner (reproduced with bike-components.de product pages and youtube.com/watch).
3. Observe Failed to capture the screenshot. Reason: Error: TIMED_OUT followed by Skipping storing the screenshot as it's empty.
4. Set CRAWLER_ENABLE_AUTOCONSENT=false, recreate, bookmark another page on the same site → the screenshot is captured and stored normally.

Expected Behaviour

Screenshots should be captured on pages with consent banners. Either take the screenshot before autoconsent interacts with the page, or wait for the page to settle after the consent click before capturing.

Screenshots or Additional Context

A/B test, same domain, only autoconsent differs

Ruled out beforehand

All measured with playwright.connectOverCDP against the same running karakeep-chrome container, same URL:

Bare screenshot of the same page through the same container: 377 ms once the page is idle. So Chrome, the image, and the page are all fine — it is specifically the state autoconsent leaves the page in.

Note on the trade-off

Disabling autoconsent is not a good workaround: with the banner left in place, extracted text dropped from ~32 KB to ~2.7 KB on comparable product pages. Users effectively have to choose between full-text extraction and screenshots.

Possibly related

• #1097 — same symptom reported in March 2025 (before autoconsent existed), closed by making the timeout configurable. Worth revisiting: the configurable timeout does not help in this class of failure, since the screenshot always consumes the whole budget.
• #2695 — connectOverCDP: Timeout 5000ms exceeded (not configurable). Seen occasionally here too; the automatic retry recovers.

Device Details

Synology DS918+ (Intel Celeron J3455, 4 cores), DSM 7, Docker Compose

Exact Karakeep Version

0.33.2

Environment Details

ghcr.io/karakeep-app/karakeep-chrome:release (Chrome 151), meilisearch v1.13.3, BROWSER_CONNECT_ONDEMAND=true, CRAWLER_JOB_TIMEOUT_SEC=120, CRAWLER_NAVIGATE_TIMEOUT_SEC=30

Debug Logs

With autoconsent on (fails):

[Crawler][25212:0] Navigating to "https://www.bike-components.de/de/crankbrothers/Mallet-E-Lace-MTB-Schuhe-p227490/"
[Crawler][25212:0] Successfully navigated ... Waiting for the page to load ...
[Crawler][25212:0] Finished waiting for the page to load.
[Crawler][25212:0] Successfully fetched the page content.
[Crawler][25212:0] Failed to capture the screenshot. Reason: Error: TIMED_OUT, consider increasing CRAWLER_SCREENSHOT_TIMEOUT_SEC
[Crawler][25212:0] Skipping storing the screenshot as it's empty.
[Crawler][25212:0] Downloaded image as assetId: eb077cd9-... (108616 bytes)
[Crawler][25212] Completed successfully

With autoconsent off (works), same site:

[Crawler][25244:0] Navigating to "https://www.bike-components.de/de/POC/Ventral-Air-MIPS-Helm-p97289/..."
[Crawler][25244:0] Finished capturing page content and a screenshot. FullPageScreenshot: false
[Crawler][25244:0] Stored the screenshot as assetId: 77bea5e2-faba-4945-ac08-53e8e5ae2097 (75953 bytes)
[Crawler][25244] Completed successfully

Have you checked the troubleshooting guide?

• I have checked the troubleshooting guide and I haven't found a solution to my problem