JavaScript tests need a structured way to start browsers and apply optional capabilities

Description

The JavaScript test harness has no model for "how should this run be configured." Environment#builder() in javascript/selenium-webdriver/testing/index.js sniffs individual environment variables inline, one if block per variable, and builds browser options as a side effect of whichever block happens to match. javascript/private/browsers.bzl feeds it hardcoded runfiles paths that no other binding uses.

The consequence is that every knob we want — which browser, which binary, headless, BiDi, and anything added later — ends up welded to an unrelated condition. This is not hypothetical; it is the current state:

• BiDi is coupled to pinned Firefox. options.enableBidi() only runs inside if ('SE_FIREFOX' in process.env), which is Linux-with-pinned-browsers only. Meanwhile build() sets the webSocketUrl capability unconditionally for Chrome/Firefox/Edge. Two mechanisms for one feature, one of them platform-dependent, and no way to turn it off.
• Browser binaries are located differently than in every other binding. JS hardcodes "mac_firefox/Firefox.app/Contents/MacOS/firefox" — a guess at the runfiles layout that is wrong under bzlmod (the real directory is +pin_browsers_extension+mac_firefox) and only resolves because locate() falls back to @bazel/runfiles repo-mapping. Every other binding uses $(rlocationpath ...).
• Pinned browsers are Linux-only, behind a stale comment (see item 2).
• A toggle keyed on the wrong thing silently does nothing. Nesting a capability inside if ('SE_CHROME' in process.env) means it never applies on macOS or under --pin_browsers=false, because those variables are only set on Linux with pinned browsers.

What we want is a way to start a browser for a test run where the browser, the binary source, and each optional capability are chosen independently — so adding the next toggle is a one-line contribution rather than a new if block wedged into builder().

Headless and BiDi below are the two concrete cases to build and validate the mechanism with. They are examples, not the goal.

Nothing here has been pushed. Item 2 is implemented on a local branch that no one else can reach, so every item is written to stand alone.

1. Give builder() a run configuration

The enabling change; everything else is small once it exists and awkward before it.

Parse the run configuration once, then have builder() compose it: the browser determines which options class and argument syntax to use, and each toggle contributes independently. Concretely, a toggle should look like this — keyed on the target browser, not on whichever env var happens to be nearby:

if (HEADLESS) {
  if (browser.name === Browser.CHROME) { /* --headless on chrome.Options */ }
  else if (browser.name === Browser.FIREFOX) { /* -headless on firefox.Options */ }
}

and it must compose with whatever earlier blocks already set, via builder.getChromeOptions() || new chrome.Options(), so it works whether or not a pinned-binary block ran.

Two traps, both hit in practice and both silent:

• Nesting a toggle inside the pinned-binary blocks makes it a no-op on macOS and under --pin_browsers=false.
• Writing one browser-agnostic block per browser looks fine for SE_CHROME/SE_FIREFOX — each is set for exactly one target, so those blocks self-select — but a toggle set for all targets then fires in every block, so a Firefox run also builds a Chrome options object.

On the bzl side, each axis should contribute its own env fragment composed with |, the way rb/spec/tests.bzl does, rather than being folded into the pinned-browser select.

2. A consistent way to point at the browser binary (implemented locally, Linux unverified)

javascript/private/browsers.bzl enables pinned browsers on Linux only; the macOS select branches are commented out above a comment claiming --experimental_inprocess_symlink_creation is needed for paths with spaces and that it breaks aspect_rules_js.

That comment is stale. macOS pinned Chrome and Firefox both work today; the build copies the space-containing paths inside Chrome.app without complaint.

What makes it work:

• Replace the hardcoded literals with $(rlocationpath @mac_firefox//:Firefox.app)/Contents/MacOS/firefox, matching Ruby, Java and Python.
• Add the single-file targets (@mac_firefox//:Firefox.app, @mac_geckodriver//:geckodriver, …) to data, since rlocationpath requires a declared prerequisite. These are the same labels java/browsers.bzl and common/browsers.bzl already pass to $(location ...).
• Add no_copy_to_bin for those files on the mocha_test. Without it analysis fails — rules_js attempts copy_js_file_to_bin_action on a file from another repo and reports "All source and data files that are not in the Bazel output tree must be in the same package as the target". no_copy_to_bin is a real js_test attribute and mocha_test already forwards **kwargs, so no macro change is needed.
• Fix a typo in the commented-out config: mac_chrome/Chrome.app)/Contents/MacOS/Chrome has a stray ).

This is what made the difference between Chrome and Firefox look browser-specific when it was not: the same pinned macOS Firefox runs fine under Ruby's wiring. Selenium Manager remains the fallback when --pin_browsers=false, which is the desired behavior.

Verified on macOS (arm64), both browsers, pinned and fallback. Not verified on Linux — --platforms=//common/remote-build:platform cannot resolve toolchains on a Mac, so the converted Linux branches were never executed. Someone needs to run bazel test //javascript/selenium-webdriver:test-window-test.js-chrome on Linux before this merges.

3. Stop copying browser bundles into the output tree

Item 2 leaves a real fragility. chrome-js and firefox-js — the js_library wrappers in common/repositories.bzl — do data = glob(["Firefox.app/**/*"]), copying the whole application bundle into bazel-out. no_copy_to_bin on the js_test does not affect that separate copy.

On macOS this fails, and fails unrecoverably:

+pin_browsers_extension+mac_firefox/BUILD.bazel:7:11: Copying file Firefox.app/Contents/CodeResources failed:
  failed to create output directory ... Firefox.app (Operation not permitted)

Once a partial copy is left behind, the bundle's signed files cannot be chmod'd or removed, and that output base stays wedged until the directory is deleted with elevated permissions.

Proposed fix: copy_data_to_bin = False on the -js js_library targets in common/repositories.bzl. Browser binaries are never import()ed, so the ESM concern that motivates the copy (item 6) does not apply to them. common/ is flagged high-risk in AGENTS.md and this affects every binding, so it wants review rather than a drive-by change.

4. Headless as an optional capability

No wiring exists today. javascript/private/browsers.bzl needs a select on @selenium//common:use_headless_browser contributing SE_HEADLESS=true to each browser's env, and builder() needs to apply it per item 1.

Two tests fail once it works, both inherent to headless rather than wiring bugs — worth deciding deliberately rather than discovering:

• test/actions_test.js clicks into a 500px fixture, but the default headless Chrome viewport measures 756x413, so the W3C in-view center point is clamped and clicks land at [250, 206] instead of [250, 250]. Either pin a window size for headless runs or have the test compute the expected point.
• test/chrome/options_test.js — the two extension tests cannot load an extension headless. Other bindings skip these through a headless guard (Ruby's :headless, Python's headless fixture); JS would need an equivalent predicate for ignore(), which is also the general answer for "this capability makes this test inapplicable."

5. BiDi as an optional capability

The messier of the two cases, and the better test of the mechanism. Today enableBidi() and the webSocketUrl capability are two separate paths, one gated on pinned Firefox and one unconditional, with no off switch — so JS never exercises the classic-only path, including across the 51 generated bidi targets.

Collapse them into one path keyed on a toggle plus the target browser, then generate -bidi target variants the way rb/spec/tests.bzl does so both modes get covered. Mocha has no equivalent of pytest's addoption, so environment variables are the right fit here; Ruby's env-vars-plus-target-variants is the closest template.

6. Background: why rules_js imposes these constraints

Worth recording so nobody re-derives it. copy_data_to_bin defaults to True because of aspect-build/rules_js#362 — Node's ESM resolver calls realPathSync without rules_js's fs monkey-patches (the CommonJS require loader's equivalent call does get them), so ESM import() follows symlinks out of runfiles into the source tree. PR #353 is specifically "mocha escapes the sandbox," since mocha calls import(url.pathToFileURL(file)) internally; PR #354 added copy_data_to_bin as the mitigation. Open since August 2022.

That copy is what forces the same-package rule: a Bazel action can only declare outputs in its own package, so cross-repo files cannot be copied by our target. The -js js_library wrappers are the documented remedy, not a local hack.

Selenium's JS tests run on mocha — the exact runner that made this the default. Turning copy_data_to_bin off wholesale is unsafe for JS sources; excluding only browser binaries (item 3) is the narrow, intended use of the escape hatch.

7. Future axes the mechanism should accommodate

Edge is a bare // Edge placeholder in testing/index.js with no target in browsers.bzl. Grid is supported by the harness through SELENIUM_REMOTE_URL / SELENIUM_SERVER_JAR but no bazel target uses it. Neither needs doing here, but item 1 should not make either harder.

8. CI gap

None of this is covered. ci-javascript.yml runs only //javascript/selenium-webdriver:small-tests — the JS browser targets are in no workflow, and no workflow passes --headless (grep -rn headless .github/workflows/ is empty). Nothing will catch a regression in any of the above automatically, which is worth weighing when deciding how much to take on.

Have you considered any alternatives or workarounds?

• Add each toggle as its own inline if in builder() — the status quo, and how BiDi ended up coupled to pinned Firefox. Rejected: each addition silently constrains the next.
• Nest toggles inside the pinned-binary blocks — smaller diff, but a silent no-op on macOS and under --pin_browsers=false. Verified broken.
• Leave JS on hardcoded runfiles literals — works on Linux today. Rejected: it is why macOS is broken and why a wiring problem presented as a browser-specific one.
• copy_data_to_bin = False on the mocha_test targets instead of the -js libraries — simpler, but disables the sandbox mitigation for all JS sources, precisely what rules_js#362 warns against for mocha.