Every managed QA vendor, including us, claims to write "good" Playwright tests.
Almost none of that claim is measurable.
So we built a scorer that doesn't take anyone's word for it — an open-source, deterministic, AI-free tool called playwright-score — and pointed it at 17 real, public Playwright suites we don't own, didn't write, and didn't curate for a good outcome.
The result: 1,214 spec files, 5,943 tests, 168,902 lines of test code, and 5,467 individual rule violations, scored on 2026-08-19 against each project's live / branch.
Every number in this post is reproducible from a single script against the public repos linked throughout — no surveys, no self-reported data, nothing from our own customers.
The headline numbers 17 public repos scored 5,467 rule violations found 30.1% of all locators are raw CSS/XPath 12 / 17 repos have a test with zero assertions The corpus We didn't pick repos to make the tool — or the industry — look good or bad.
The corpus mixes heavily-engineered platforms (Supabase, Grafana, Mattermost, n8n) with smaller, less mature projects found by searching for real usage.
Every repo is scored on the exact subdirectory that holds its Playwright suite, via a fresh, shallow, sparse clone — nothing is vendored or cached.
Repo Score Grade Files Tests Findings Playwright (own TodoMVC example) 98/100 A 24 24 4 Supabase 98/100 A 31 271 167 freeCodeCamp 97/100 A 89 372 187 Documenso 96/100 A 125 1109 579 Storybook 96/100 A 6 25 12 dub 96/100 A 17 147 27 n8n 95/100 A 256 1047 39 novu 93/100 A 2 2 3 Grafana 92/100 A 213 672 744 PostHog 91/100 A 42 124 261 Immich 90/100 A 13 43 39 Mattermost 90/100 A 284 1157 1476 sencho 90/100 A 26 177 263 cal.com 85/100 B 53 278 1215 livecodes 78/100 C 14 280 140 openplayerjs 77/100 C 8 77 36 TheCyberHub 73/100 C 11 138 275 Run on 2026-08-19 with playwright-score sqs-v1, profile, threshold
80.
Reproduce it yourself with scripts/validate-corpus.sh in the GitHub repo.
What's actually wrong with real Playwright suites Grades hide the interesting part.
We also kept every individual finding — 5,467 of them — and ranked which specific anti-patterns show up most often across the corpus.
This is the part that isn't on our own tool's landing page.
Rule Occurrences Repos affected What it means 3,287 16 / 17 Raw page.locator()/frame.locator() CSS or XPath instead of role-based locators 572 8 / 17 Legacy page.click(selector) style instead of Locator API 450 11 / 17 Hard-coded sleeps (waitForTimeout) instead of condition-based waits 219 12 / 17 Assertions that don't auto-retry against the live DOM 194 8 / 17 waitForLoadState('networkidle') — deprecated, unreliable in Playwright 144 9 / 17 bypassing actionability checks 103 8 / 17 Manual waitForSelector instead of auto-waiting locators 102 12 / 17 A test block that runs and passes without asserting anything 71 10 / 17 Spec files large enough to hurt reviewability and maintenance 70 3 / 17 Un-awaited async Playwright calls — silent race conditions Two patterns dominate everything else.
Locator anti-patterns account for 4,067 of the 5,467 total findings — 74% of every violation in the corpus — and alone is 60% of all findings, present in 16 of the 17 repos.
This is not a niche mistake.
It's the default way most teams still write Playwright selectors, three years after Playwright's own docs started recommending role-based locators over raw CSS.
Zoomed out across every locator call in the corpus — not just findings, every single // vs. / call, 17,118 of them — 69.9% are role-based and 30.1% are raw selectors.
Two repos in the corpus, livecodes and openplayerjs, use zero role-based locators: every single locator in both suites is a raw CSS or XPath selector.
Both fail the default threshold.
A B-grade suite can hide more findings than an A cal.com's suite triggers 1,215 individual findings across only 53 files — roughly 23 per file, the highest density in the corpus — and still scores an 85 (B), passing the default threshold.
Mattermost, in contrast