[Bug]: analyze_landing.py exits 0 and prints a full all-FAIL audit when the page was never fetched

Operating System

Windows

Python Version

3.12.10

Command That Failed

python scripts/analyze_landing.py https://example.com --timeout 45000

Full Error Output

$ python scripts/analyze_landing.py https://example.com --timeout 45000

Landing Page Quality Analysis
==================================================

URL: https://example.com

Performance:
  LCP: Nonems (N/A)
  CLS: None (N/A)
  TTFB: Nonems

Content:
  Title: None
  H1: MISSING
  Words: 0

Conversion Elements:
  CTA Above Fold: N
  Form: N
  Phone: N
  Chat: N

Schema: None

Audit Grades:
  [FAIL] G60_relevance
  [FAIL] G61_schema
  [FAIL] cta_above_fold
  [FAIL] mobile_responsive

Error: Browser request blocked: a verified egress-sandbox attestation document is required.

$ echo $?
0

# Same behaviour with --json: exit code 0, word_count 0, all four grades FAIL,
# with the real cause present only in the "error" key.

Ad Platform and Account Context

No ad platform involved — this is the landing-page analyzer script itself, reproducible against any URL.

Root cause

analyze_landing() converts every failure into a data field rather than an error:

• L101–103 — except ValueError sets result["error"] and returns the default result (this is the blocked-egress path)
• L289–290 — except PlaywrightTimeout sets result["error"], falls through to return result at L294
• L291–292 — bare except Exception does the same

main() then grades that empty result unconditionally and prints the error last, at L407, without ever raising SystemExit:

if result["error"]:
    print(f"\nError: {result['error']}")

So the function returns normally and the process exits 0. The four FAIL grades are scored against an all-empty default dict, not against the page.

Notably main() already handles the other failure mode correctly — a malformed attestation document exits 1 at L363–365 — so the inconsistency looks unintentional rather than by design.

Impact

• Any wrapper, CI step, or agent checking the exit code treats a hard failure as a successful audit.
• The printed report is plausible rather than obviously broken: "no H1, no schema, no CTA above fold" reads exactly like a genuine finding about a poor landing page. The page I first hit this on actually has a title, an <h1>, ~500 words, Product + Organization JSON-LD and a working CTA — none of which was read.
• In an agent-driven workflow this is the sharp edge: fabricated all-FAIL findings can flow into a client-facing deliverable with no signal that measurement never happened.

Environment

• claude-ads 2.0.1
• Windows 11, CPython 3.12.10, managed venv from requirements.lock (runtime-windows-cp312)
• playwright 1.61.0, Chromium 149.0.7827.55
• Reproduced both with and without --json

Scope note: the timeout and generic-exception paths (L289–292) are identified by code inspection rather than direct reproduction — the attestation gate short-circuits before DNS resolution, so I could not exercise them independently. The confirmed reproduction is the blocked-egress path at L101–103.

Expected Behavior

A run that never fetched the page should be distinguishable from a run that fetched it and scored it badly — both to a human reading the output and to a caller checking the exit code.

Concretely:

1. Exit non-zero when result["error"] is set. A blocked, timed-out, or errored run is a failure, not a completed audit.
2. Suppress or clearly mark the grade block when no page was loaded. Grading a page that was never fetched produces misleading output regardless of the exit code — this is the part that turns a failure into a plausible-looking finding.
3. Apply the same guard to --json, and consider omitting grades entirely on error so structured consumers cannot mistake defaults for measurements.

Something like this, placed before the grade rendering:

if result["error"]:
    print(f"\nError: {result['error']}", file=sys.stderr)
    raise SystemExit(1)

Routing the message to stderr as well would match the convention already used for the attestation-parse failure at L363–365.

Happy to open a PR if that would be useful.