[ Feature]: Remove dependencies on public URLs from tests
Description
Several tests navigate to or connect to real public sites. These tests fail or flake when the site changes, is slow, is blocked by CI networking, or when the browser changes behavior on untrusted networks (we already have xfail_chrome markers for HTTP to HTTPS auto-upgrade on http://example.com). They also prevent running the suites with network access disabled.
A larger set of tests uses real registrable domains (example.com, google.com, test.com, etc.) purely as string data. Those never hit the network, but they are inconsistent and make it hard to audit which tests actually need network access.
This is breaking trunk CI now
Python / Browser Tests (chrome-bidi, windows) failed on trunk (4732af16c5). The only failures in the job are the two copies of one test, each failing all 3 attempts, so this is deterministic and not a flake:
//py:test/selenium/webdriver/common/bidi/browser_tests-chrome-bidi//py:test/selenium/webdriver/common/_bidi/browser_tests-chrome-bidi
FAILED py\test\selenium\webdriver\common\bidi\browser_tests.py::test_create_user_context_with_direct_proxy[chrome]
AssertionError: assert 'example domain' in 'this domain is for use in documentation examples without needing permission. this is not a service, avoid relying on it for testing and monitoring purposes.\nlearn more'Underlying cause: nothing changed in Selenium, the browser, or the driver. test_create_user_context_with_direct_proxy loads http://example.com/ and asserts the body text contains "example domain". IANA changed the page (Last-Modified: Mon, 28 Sep 2026 16:19:23 GMT, the day before the failing run) and removed the <h1>Example Domain</h1>. The body is now:
<body><p>This domain is for use in documentation examples without needing permission. This is not a service, avoid relying on it for testing and monitoring purposes.</p><a href=https://iana.org/help/example-domains>Learn more</a><script src=/s.js></script></body>"Example Domain" survives only in <title>, which is not part of the body text. The new page text itself says not to rely on it for testing.
The immediate fix is to point this test at a page on the local test server. Matching the new text or the title would only reset the clock until the next change. The rest of this issue covers the other tests with the same exposure.
Tests that reach the public network
| Binding | Location | Host | Notes |
|---|---|---|---|
| Python | py/test/selenium/webdriver/common/bidi/browser_tests.py (test_create_user_context_with_accept_insecure_certs, ..._with_direct_proxy, ..._with_proxy_and_accept_insecure_certs) |
self-signed.badssl.com, example.com |
Asserts on third-party page content ("example domain", badssl h1) |
| Python | py/test/selenium/webdriver/common/_bidi/browser_tests.py |
same | Near-duplicate of the file above; confirm whether both directories should exist |
| Python | py/test/selenium/webdriver/common/bidi/browser_tests.py (..._manual_proxy_all_params) |
example.com |
Goes through the local fake proxy, but already xfail on Chrome because of HTTPS upgrade |
| Python | py/test/selenium/webdriver/remote/remote_hub_connection_tests.py |
wrong.host.badssl.com |
Asserts on the TLS error text from a third-party host |
| Python | py/test/selenium/webdriver/common/example2.py |
www.google.com |
Searches Google and asserts on results; nothing references it, candidate for deletion |
| .NET | dotnet/test/webdriver/NetworkInterceptionTests.cs (TransformNetworkResponse) |
www.selenium.dev |
Response is replaced by the handler, so a local page works equally well |
| .NET | dotnet/test/webdriver/BiDi/Input/CombinedInputActionsTests.cs (Paint) |
kleki.com |
[Test] is commented out; delete or port to a local page |
| Ruby | rb/spec/integration/selenium/webdriver/manager_spec.rb |
saucelabs.com subdomains, watir.com, selenium.dev |
Guarded with skip_unless: {driver: :none} so they never run; they need subdomains or HTTPS |
| Rust | rust/tests/mirror_tests.rs |
registry.npmmirror.com |
Third-party mirror; Selenium Manager tests need network by design, but this adds a dependency on a host we do not control |
| Shared pages | common/src/web/mouse_interaction.html |
selenium.dev |
Loads jQuery and jQuery UI CSS/JS from selenium.dev instead of the local copies |
| Java | java/test/org/openqa/selenium/PageLoadingTest.java |
www.thisurldoesnotexist.comx |
Not a reserved TLD; currently @NotYetImplemented for all browsers |
| Java | java/test/org/openqa/selenium/grid/distributor/DistributorNodeAvailabilityTest.java |
does.not.exist |
Not a reserved TLD |
Real domains used only as data (no network access)
Not failures today, but worth normalizing:
google.com: JavaEndToEndTestandGridSupplier.main(spoofed session, never loads), JStest/lib/http_test.js,webdriver_test.js,until_test.js, Ruby IE options and Firefox profile specs, Pythonexample2.pytest.com,some.host.com,cheese.com,my.domain.com,selenium2.ru,idp.com,foo.com,boo.com,yes.com,froogle.com,another.server.com,get.com,navigate-to.com,http_proxy.com,https_proxy.com,tracking.com,other.com,proxy.org,pac.url,a.base.url: mocks, proxy config, and parsing tests across all bindingsexample.com/example.orgin unit tests: reserved by RFC 2606 and acceptable as data, as long as nothing resolves or loads itHttpClientTestBase.shouldAllowUrlsWithSchemesToBeUsedcreates a client forhttp://example.com(the code comment already calls it a terrible choice); the request itself goes to the local server
Already correct, no change needed: *.test, *.example, and *.invalid hosts (not_a_valid_url.test, invalid-domain-that-does-not-exist.test, doesnotexist.invalid, rp.example, idp.example, grid.example), and URLs in comments, skip reasons, and error message assertions.
Proposed changes
- Serve everything from the local test web server.
- Insecure cert tests: use the test server's existing self-signed HTTPS endpoint instead of badssl.com.
- Proxy tests: navigate to a reserved hostname (for example
http://proxied.test/) that only resolves via the fake proxy, and use a local page for the direct-proxy case. - Cookie domain and secure cookie tests: add subdomain and HTTPS support to the test environment (for example
*.localhostaliases) so the Ruby specs can run instead of being permanently skipped. TransformNetworkResponse: use a local page.mouse_interaction.html: reference the jQuery files already incommon/src/web.
- Replace unresolvable-host URLs with reserved names (
.invalidor.test). - Replace real domains used as data with reserved ones (
example.com,*.example,*.test). - Delete dead code:
example2.py(andgoogle_one_boxif unused), the disabledPainttest. - Decide on the Rust mirror test: keep with an explicit note that it depends on a third party, or point it at a mirror we control.
- Add a lint check that fails on new non-reserved hosts in test directories, with an allowlist for comments, bug tracker links, and Selenium Manager tests.
- Once clean, audit
requires-networktags so only targets that truly need it (browser and driver downloads, Selenium Manager) keep it.
Have you considered any alternatives or workarounds?
Marking the affected tests as flaky or skipping them in CI hides the problem and loses coverage. Keeping public URLs but retrying does not help with content changes or blocked networks.
Source: SeleniumHQ/selenium