fetch_page.py misses ~70% of content on JS-hydrated sites — recommend Playwright fallback
Author: getrocketsalesCreated May 9, 2026Updated May 9, 2026
geo-seo-claude's default page fetcher (scripts/fetch_page.py) uses raw requests + lxml parsing, which only captures the SSR shell on JS-hydrated sites (Next.js, React, Vue, Astro). On a real-world medical site I audited, this caused the tool to miss 4 complete Physician JSON-LD schemas, 6 named providers with credentials, and ~70% of body content — producing an audit score 30 points below the correct value. Because requirements.txt already pulls in Playwright and the installer already runs playwright install chromium, the fix is essentially free: detect hydration, fall back to a rendered fetch. A drop-in replacement (~80 lines) is included in the issue.
Source: zubair-trabzada/geo-seo-claude