PageSpeed Insights / Lighthouse synthetic performance results may not always measure the same page lifecycle experienced by a normal first-time user.
FAQ
- Yes, my issue is not about variability or throttling.
- Yes, my issue is not about a specific accessibility audit (file with axe-core instead).
- Yes, my issue is not answered by other FAQs.
URL
What happened?
Summary
PageSpeed Insights / Lighthouse synthetic performance results may not always measure the same page lifecycle experienced by a normal first-time user.
Some optimization software can identify requests associated with performance-testing clients by User-Agent and/or dedicated Guest Mode logic, and then keep those clients on a specially optimized execution path.
A concrete example is LiteSpeed Cache for WordPress Guest Mode / Guest Optimization.
This is not a report about normal Lighthouse score variability or throttling differences. The concern is that the system under test can detect the measurement client and intentionally or unintentionally expose a different execution path to it.
Why this matters
A benchmark should ideally not materially alter the behavior of the system merely by being identifiable as the benchmark.
If PageSpeed Insights can be detected as PageSpeed Insights, a website optimization layer can:
- serve a different cached variant;
- apply optimizations that are otherwise disabled;
- delay or avoid JavaScript execution differently;
- prevent transition from a temporary “Guest” representation to the normal visitor representation;
- consequently produce LCP, TBT and other laboratory metrics that do not represent the lifecycle followed by a regular browser visitor.
This may produce excellent synthetic Lighthouse scores while the actual browser execution path is substantially heavier.
Concrete implementation: LiteSpeed Cache for WordPress
LiteSpeed's own documentation describes Guest Mode as follows:
A default cached version is initially served to a visitor. After the HTML is loaded, an AJAX request normally causes the visitor to receive the “correct” version, considering cache varies, ESI and the site's ordinary optimizations.
Guest Optimization goes further: LiteSpeed states that the maximum level of page and image optimization is applied to this default page even when those optimization features are otherwise disabled.
The documentation also states that qualifying requests generally include bots, certain User-Agents and a user's first visit.
Most importantly, LiteSpeed explicitly documents that Guest Optimization can greatly improve page-speed scores because services such as GTmetrix and PageSpeed Insights will receive a cached, highly optimized page.
Relevant code behavior
The current LiteSpeed Cache WordPress source contains an always_guest() mechanism.
The Guest class loads Guest Mode User-Agent patterns and checks the incoming HTTP User-Agent against them.
When always_guest() matches during the Guest vary update, the code exits without requesting the normal reload/transition.
For ordinary Guest Mode operation, when no permanent Guest condition applies, the code sets the Guest vary and returns:
{"reload":"yes"}
This creates a material distinction:
Normal visitor
Initial Guest representation → Guest vary update → reload/transition → normal visitor-specific representation
Client matching always_guest()
Initial Guest representation
→ Guest vary update
→ always_guest() matches
→ no normal reload/transition
This means an identifiable measurement client may stay on a different execution path.
Expected behavior
PageSpeed Insights should either:
- be sufficiently indistinguishable from the equivalent real Chrome navigation that server-side software cannot trivially identify it as a benchmark client;
or
- detect/report when the tested origin is serving a materially different response or lifecycle specifically to Lighthouse/PageSpeed;
or
- clearly flag the synthetic result as potentially non-representative when benchmark-specific response differentiation is detected.
Ideally PSI could perform a comparison using at least two equivalent navigation identities and detect material differences in:
- response HTML;
- response headers;
- cache variants;
- JavaScript resources;
- network dependency graph;
- subsequent reload/navigation behavior;
- total transferred bytes;
- DOM state;
- execution tasks;
- LCP candidate;
- TBT / long tasks.
Suggested reproduction methodology
A controlled test could be performed against a WordPress installation using LiteSpeed Cache with:
- Guest Mode enabled;
- Guest Optimization enabled;
- otherwise ordinary production settings.
Run equivalent tests with:
A. the exact identity used by PageSpeed Insights / Lighthouse infrastructure;
B. a normal Chrome client with equivalent viewport, device emulation, cache state, throttling and first-visit conditions.
Record:
- initial HTML;
- HTTP headers;
- cookies;
- Guest vary requests;
- subsequent navigation/reload;
- loaded JS/CSS;
- request waterfall;
- DOM after load;
- LCP;
- TBT;
- CLS;
- execution trace.
The relevant question is not whether the scores differ slightly.
The test should determine whether identification of the benchmark itself causes a different application state or resource/execution path.
Security / integrity implication
This mechanism does not necessarily imply malicious intent by the optimization vendor or website operator.
There are legitimate uses for cache variants, bot handling and first-visitor optimization.
However, from a measurement-integrity perspective, a synthetic benchmark that is identifiable by the system under test creates an opportunity for benchmark-specific behavior.
The same general problem applies beyond LiteSpeed Cache: any CDN, reverse proxy, optimization plugin, WAF or application can potentially identify Lighthouse/PSI and alter its behavior.
Therefore this appears to be a PageSpeed Insights / Lighthouse benchmark-integrity issue rather than solely a LiteSpeed-specific issue.
Suggested mitigation
One possible mitigation would be to introduce an audit specifically for response divergence.
For example:
Benchmark-specific response behavior detected
The audit could compare a control navigation against the standard Lighthouse navigation and warn when substantial differences are detected in HTML, resource loading or page lifecycle.
Alternatively, PageSpeed Insights could reduce easily identifiable characteristics of its synthetic navigation where technically possible.
Additional evidence
An independent technical analysis published by Managed Server examines this exact behavior in LiteSpeed Cache and compares PageSpeed Insights results with browser-based Lighthouse testing.
The analysis reports substantial differences between the two execution contexts and traces the behavior to the Guest Mode logic.
This report is being submitted primarily because the underlying issue is broader than one plugin:
A benchmark whose identity is detectable can cause the software being benchmarked to change what the benchmark measures.
What did you expect?
Summary
PageSpeed Insights / Lighthouse synthetic performance results may not always measure the same page lifecycle experienced by a normal first-time user.
Some optimization software can identify requests associated with performance-testing clients by User-Agent and/or dedicated Guest Mode logic, and then keep those clients on a specially optimized execution path.
A concrete example is LiteSpeed Cache for WordPress Guest Mode / Guest Optimization.
This is not a report about normal Lighthouse score variability or throttling differences. The concern is that the system under test can detect the measurement client and intentionally or unintentionally expose a different execution path to it.
Why this matters
A benchmark should ideally not materially alter the behavior of the system merely by being identifiable as the benchmark.
If PageSpeed Insights can be detected as PageSpeed Insights, a website optimization layer can:
- serve a different cached variant;
- apply optimizations that are otherwise disabled;
- delay or avoid JavaScript execution differently;
- prevent transition from a temporary “Guest” representation to the normal visitor representation;
- consequently produce LCP, TBT and other laboratory metrics that do not represent the lifecycle followed by a regular browser visitor.
This may produce excellent synthetic Lighthouse scores while the actual browser execution path is substantially heavier.
Concrete implementation: LiteSpeed Cache for WordPress
LiteSpeed's own documentation describes Guest Mode as follows:
A default cached version is initially served to a visitor. After the HTML is loaded, an AJAX request normally causes the visitor to receive the “correct” version, considering cache varies, ESI and the site's ordinary optimizations.
Guest Optimization goes further: LiteSpeed states that the maximum level of page and image optimization is applied to this default page even when those optimization features are otherwise disabled.
The documentation also states that qualifying requests generally include bots, certain User-Agents and a user's first visit.
Most importantly, LiteSpeed explicitly documents that Guest Optimization can greatly improve page-speed scores because services such as GTmetrix and PageSpeed Insights will receive a cached, highly optimized page.
Relevant code behavior
The current LiteSpeed Cache WordPress source contains an always_guest() mechanism.
The Guest class loads Guest Mode User-Agent patterns and checks the incoming HTTP User-Agent against them.
When always_guest() matches during the Guest vary update, the code exits without requesting the normal reload/transition.
For ordinary Guest Mode operation, when no permanent Guest condition applies, the code sets the Guest vary and returns:
{"reload":"yes"}
This creates a material distinction:
Normal visitor
Initial Guest representation → Guest vary update → reload/transition → normal visitor-specific representation
Client matching always_guest()
Initial Guest representation
→ Guest vary update
→ always_guest() matches
→ no normal reload/transition
This means an identifiable measurement client may stay on a different execution path.
Expected behavior
PageSpeed Insights should either:
- be sufficiently indistinguishable from the equivalent real Chrome navigation that server-side software cannot trivially identify it as a benchmark client;
or
- detect/report when the tested origin is serving a materially different response or lifecycle specifically to Lighthouse/PageSpeed;
or
- clearly flag the synthetic result as potentially non-representative when benchmark-specific response differentiation is detected.
Ideally PSI could perform a comparison using at least two equivalent navigation identities and detect material differences in:
- response HTML;
- response headers;
- cache variants;
- JavaScript resources;
- network dependency graph;
- subsequent reload/navigation behavior;
- total transferred bytes;
- DOM state;
- execution tasks;
- LCP candidate;
- TBT / long tasks.
Suggested reproduction methodology
A controlled test could be performed against a WordPress installation using LiteSpeed Cache with:
- Guest Mode enabled;
- Guest Optimization enabled;
- otherwise ordinary production settings.
Run equivalent tests with:
A. the exact identity used by PageSpeed Insights / Lighthouse infrastructure;
B. a normal Chrome client with equivalent viewport, device emulation, cache state, throttling and first-visit conditions.
Record:
- initial HTML;
- HTTP headers;
- cookies;
- Guest vary requests;
- subsequent navigation/reload;
- loaded JS/CSS;
- request waterfall;
- DOM after load;
- LCP;
- TBT;
- CLS;
- execution trace.
The relevant question is not whether the scores differ slightly.
The test should determine whether identification of the benchmark itself causes a different application state or resource/execution path.
Security / integrity implication
This mechanism does not necessarily imply malicious intent by the optimization vendor or website operator.
There are legitimate uses for cache variants, bot handling and first-visitor optimization.
However, from a measurement-integrity perspective, a synthetic benchmark that is identifiable by the system under test creates an opportunity for benchmark-specific behavior.
The same general problem applies beyond LiteSpeed Cache: any CDN, reverse proxy, optimization plugin, WAF or application can potentially identify Lighthouse/PSI and alter its behavior.
Therefore this appears to be a PageSpeed Insights / Lighthouse benchmark-integrity issue rather than solely a LiteSpeed-specific issue.
Suggested mitigation
One possible mitigation would be to introduce an audit specifically for response divergence.
For example:
Benchmark-specific response behavior detected
The audit could compare a control navigation against the standard Lighthouse navigation and warn when substantial differences are detected in HTML, resource loading or page lifecycle.
Alternatively, PageSpeed Insights could reduce easily identifiable characteristics of its synthetic navigation where technically possible.
Additional evidence
An independent technical analysis published by Managed Server examines this exact behavior in LiteSpeed Cache and compares PageSpeed Insights results with browser-based Lighthouse testing.
What have you tried?
Summary
PageSpeed Insights / Lighthouse synthetic performance results may not always measure the same page lifecycle experienced by a normal first-time user.
Some optimization software can identify requests associated with performance-testing clients by User-Agent and/or dedicated Guest Mode logic, and then keep those clients on a specially optimized execution path.
A concrete example is LiteSpeed Cache for WordPress Guest Mode / Guest Optimization.
This is not a report about normal Lighthouse score variability or throttling differences. The concern is that the system under test can detect the measurement client and intentionally or unintentionally expose a different execution path to it.
Why this matters
A benchmark should ideally not materially alter the behavior of the system merely by being identifiable as the benchmark.
If PageSpeed Insights can be detected as PageSpeed Insights, a website optimization layer can:
- serve a different cached variant;
- apply optimizations that are otherwise disabled;
- delay or avoid JavaScript execution differently;
- prevent transition from a temporary “Guest” representation to the normal visitor representation;
- consequently produce LCP, TBT and other laboratory metrics that do not represent the lifecycle followed by a regular browser visitor.
This may produce excellent synthetic Lighthouse scores while the actual browser execution path is substantially heavier.
Concrete implementation: LiteSpeed Cache for WordPress
LiteSpeed's own documentation describes Guest Mode as follows:
A default cached version is initially served to a visitor. After the HTML is loaded, an AJAX request normally causes the visitor to receive the “correct” version, considering cache varies, ESI and the site's ordinary optimizations.
Guest Optimization goes further: LiteSpeed states that the maximum level of page and image optimization is applied to this default page even when those optimization features are otherwise disabled.
The documentation also states that qualifying requests generally include bots, certain User-Agents and a user's first visit.
Most importantly, LiteSpeed explicitly documents that Guest Optimization can greatly improve page-speed scores because services such as GTmetrix and PageSpeed Insights will receive a cached, highly optimized page.
Relevant code behavior
The current LiteSpeed Cache WordPress source contains an always_guest() mechanism.
The Guest class loads Guest Mode User-Agent patterns and checks the incoming HTTP User-Agent against them.
When always_guest() matches during the Guest vary update, the code exits without requesting the normal reload/transition.
For ordinary Guest Mode operation, when no permanent Guest condition applies, the code sets the Guest vary and returns:
{"reload":"yes"}
This creates a material distinction:
Normal visitor
Initial Guest representation → Guest vary update → reload/transition → normal visitor-specific representation
Client matching always_guest()
Initial Guest representation
→ Guest vary update
→ always_guest() matches
→ no normal reload/transition
This means an identifiable measurement client may stay on a different execution path.
Expected behavior
PageSpeed Insights should either:
- be sufficiently indistinguishable from the equivalent real Chrome navigation that server-side software cannot trivially identify it as a benchmark client;
or
- detect/report when the tested origin is serving a materially different response or lifecycle specifically to Lighthouse/PageSpeed;
or
- clearly flag the synthetic result as potentially non-representative when benchmark-specific response differentiation is detected.
Ideally PSI could perform a comparison using at least two equivalent navigation identities and detect material differences in:
- response HTML;
- response headers;
- cache variants;
- JavaScript resources;
- network dependency graph;
- subsequent reload/navigation behavior;
- total transferred bytes;
- DOM state;
- execution tasks;
- LCP candidate;
- TBT / long tasks.
Suggested reproduction methodology
A controlled test could be performed against a WordPress installation using LiteSpeed Cache with:
- Guest Mode enabled;
- Guest Optimization enabled;
- otherwise ordinary production settings.
Run equivalent tests with:
A. the exact identity used by PageSpeed Insights / Lighthouse infrastructure;
B. a normal Chrome client with equivalent viewport, device emulation, cache state, throttling and first-visit conditions.
Record:
- initial HTML;
- HTTP headers;
- cookies;
- Guest vary requests;
- subsequent navigation/reload;
- loaded JS/CSS;
- request waterfall;
- DOM after load;
- LCP;
- TBT;
- CLS;
- execution trace.
The relevant question is not whether the scores differ slightly.
The test should determine whether identification of the benchmark itself causes a different application state or resource/execution path.
Security / integrity implication
This mechanism does not necessarily imply malicious intent by the optimization vendor or website operator.
There are legitimate uses for cache variants, bot handling and first-visitor optimization.
However, from a measurement-integrity perspective, a synthetic benchmark that is identifiable by the system under test creates an opportunity for benchmark-specific behavior.
The same general problem applies beyond LiteSpeed Cache: any CDN, reverse proxy, optimization plugin, WAF or application can potentially identify Lighthouse/PSI and alter its behavior.
Therefore this appears to be a PageSpeed Insights / Lighthouse benchmark-integrity issue rather than solely a LiteSpeed-specific issue.
Suggested mitigation
One possible mitigation would be to introduce an audit specifically for response divergence.
For example:
Benchmark-specific response behavior detected
The audit could compare a control navigation against the standard Lighthouse navigation and warn when substantial differences are detected in HTML, resource loading or page lifecycle.
Alternatively, PageSpeed Insights could reduce easily ide
Source: GoogleChrome/lighthouse