Self-hosted Playwright response omits observed final navigation URL
The self-hosted Playwright service follows an HTTPS redirect and returns the destination's HTML and status, but omits the final navigation URL. The API's Playwright engine then assigns its input or rewritten URL to the result, rather than consuming an observation from the browser.
Verified on the published Linux ARM64 images built from f75a8d40b103129f56f947418bfa06a04a9ad5b0:
- API:
ghcr.io/firecrawl/firecrawl@sha256:54e5854300deb312a0ff22b27bb1a90b09967f9a5d0f64d853fccd0b1519f926 - Playwright:
ghcr.io/firecrawl/playwright-service@sha256:4e2a086f8ccc33a3acc0cd2ad3ea9f6173c441c9380581aa00aae4643e4c85f3
The API's BUILD_SHA and the browser api.ts bytes match that revision. Docker Engine 29.8.0, native Linux ARM64 on Docker Desktop.
From the API container, POST to the internal browser endpoint:
const r = await fetch('http://playwright-service:3000/scrape', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
url: 'https://httpbin.org/redirect-to?url=https%3A%2F%2Fexample.org%2F',
wait_after_load: 500,
timeout: 30000,
skip_tls_verification: false
})
});
const data = await r.json();
console.log(r.status, Object.keys(data), data.pageStatusCode,
data.content.includes('Example Domain'));Actual: HTTP 200, keys content, pageStatusCode, contentType, target status 200, and Example Domain content. There is no final URL or redirect/navigation observation.
Relevant source:
apps/playwright-service-ts/api.ts: response omitspage.url()and final main-document response identity.apps/api/src/scraper/scrapeURL/engines/playwright/index.ts: returnsurl: meta.rewrittenUrl ?? meta.url(with the redirect TODO).
Expected: return and propagate an explicit observed final document URL, including when it equals the requested URL. For JavaScript navigation during the wait interval, the status/content type should correspond to the final main-document response, or the result should explicitly say that observation is unavailable. A redirect/navigation chain and separate response URL would make same-document history changes distinguishable from network navigation.
Impact: a host cannot use the current response to establish the captured document's destination identity. The originally requested URL, canonical metadata, or an unrelated pre-fetch does not establish the browser's final location. Hosts requiring this evidence must refuse instead of inferring it.
No workspace or source data is needed for this reproduction; the two destinations are public demonstration sites. The installed browser was unmodified. I checked existing final-URL/Playwright redirect reports before filing this reproduction.
Source: firecrawl/firecrawl