Headless lacks a near-duplicate page explore budget
Author: Mzack9999Created Jul 17, 2026Updated Jul 27, 2026
URL filters (-fsu) and content similarity help, but the headless graph can still fully explore many near-duplicate locations (product templates, calendar cells). That wastes browser time without new doors.
Add a simhash cluster budget on page states (ExplosionBudget / ExplosionHamming) so only the first N near-dupes are fully explored.
Source: projectdiscovery/katana