Scheduled coverage still missing after #4848: native macOS crash-report assertions and updater download/install
Context
#4849 makes the scheduled Full regression report journeys this Linux lane cannot run as skipped: lane cannot satisfy prerequisites, and #4848 owns building the enterprise binary in that lane so the six packaged journeys execute. Even once #4848 lands, two coverage dimensions of those journeys remain unexecuted anywhere automatically, because unpacked Linux --dir coverage is not release/installer/native-macOS coverage. The second-opinion review of #4849 named both; this issue owns them.
1. Native macOS crash-report assertions (desktop-quit-path)
evals/specs/desktop-quit-path.e2e.test.ts:76–90 asserts "no crash report written" only when CRASH_REPORTS_OBSERVABLE (process.platform === "darwin", evals/worlds/desktop-quit-path.ts:31). On Linux the branch records Crash-report check skipped — needs: macOS DiagnosticReports, and only the exit code/signal is the witness. The negative half of the quit-path claim (#4787) — a clean quit leaves nothing in ~/Library/Logs/DiagnosticReports — is therefore never asserted by CI.
2. Updater download/install, not just the attempt (packaged-preactivation-updater)
evals/specs/packaged-preactivation-updater.e2e.test.ts:72–90 asserts that an activated enterprise install starts an update check (activity.checks > 0). Its own comment states why nothing more is asserted: the Den is unreachable, the network may refuse the manifest, and an unpacked Linux directory cannot self-update. So "the update was downloaded and applied, and the same profile boots afterwards" is not covered by this spec. released-enterprise-activated covers an in-place update path but only with a released baseline binary, which no lane provides (#4848).
Proposal
- A macOS execution lane (self-hosted or hosted macOS runner, or a signed Daytona macOS host if/when available) that packages the enterprise flavor as a real installable artifact (
electron-builder --mac), runsdesktop-quit-pathwithCRASH_REPORTS_OBSERVABLE === true, and runs an updater journey against a controlled update feed (the world already exposesupdaterActivity(); add a local feed serving a newer signed build) asserting download → install → relaunch → same profile boots. - Keep the Linux assertions as they are; do not weaken them to make the macOS lane easier.
Acceptance criteria (measurable)
- A scheduled run on macOS lists
desktop-quit-pathas passed with the evidence lineNo crash report appears in … DiagnosticReportsrecorded (not theCrash-report check skippedline), for both fresh and activated installs and both stimuli (SIGTERM, Browser.close). - An updater journey on a packaged, installable artifact records passed with evidence that
downloads > 0, the update was applied (version string changes across relaunch), and the pre-update profile still boots — not onlychecks > 0. - Both journeys declare their prerequisites in
TestNeedsvocabulary (needs: { platform: "darwin" }, any feed/env var) andjourney-catalog.mjsmirrors them so the parity guard injourney-ci.test.mjspasses; the Linux lane keeps reporting them as skipped with the reason, never as passed. - Run URLs for both are linked here.
Refs
- #4849 — reclassification PR; second-opinion review (comment) names these two dimensions
- #4848 — packaged enterprise binary in the scheduled lane (prerequisite for the Linux half)
- #4787 (quit path), #4809 (egress gate), #4805
Source: different-ai/openwork