Make the website Playwright smoke test verify that the page actually loaded
The website has one Playwright test, page should load without errors. It installs a pageerror listener and calls page.goto, but it never checks the navigation response or anything on the page.
page.goto doesn't fail just because the server returned a 404 or 500. So we can serve an error page and still get a green E2E test, which is the main thing this smoke test is supposed to catch.
There's a second small race in the workflow. http-server is started in the background, then the next step starts Playwright immediately. There isn't a readiness check or Playwright webServer config connecting those two.
I think we should keep this as a small smoke test, but make it real:
- assert that the response from
page.gotois OK - assert one stable thing from the rendered homepage, probably the title or a main heading
- have Playwright start/wait for the server, or explicitly wait until port 8080 is ready
That gives us an actual "the built site came up" check without turning this into a big browser suite.
Source: runatlantis/atlantis