Last year I renamed a field in an API response. became , matching the rest of the codebase.
Every test passed.
Review approved it in about four minutes.
Three days later a mobile team filed a P1.
Their client had been reading for eighteen months.
Nothing in the pipeline was broken.
The tests tested our code, and our code was fine.
What broke was the contract between us and someone else — and nothing in CI was looking at that.
I've spent the past year building tooling around this problem, and I used most of the existing tools along the way.
This is what I learned about where each one actually wins.
Disclosure before we start: I'm the author of SpecShield, one of the tools below.
I'll be straight about where the free alternatives beat it, because for a lot of teams they will.
If all you need is what oasdiff does, use oasdiff — it's excellent and it costs nothing. "Contract testing" is four different jobs Most of the confusion in this space comes from one phrase covering four unrelated problems.
Once you separate them, tool choice gets obvious.
Job 1 — Did this change break anything at all?
Diff two OpenAPI specs, classify what changed, flag what's backwards-incompatible.
Removed endpoints, narrowed types, newly-required request fields.
Job 2 — Is this spec any good?
Linting.
Missing descriptions, inconsistent casing, no , auth described in ways that will bite you later.
Job 3 — Will this break a specific consumer I know about?
Different question entirely.
Removing an endpoint nobody calls is free.
Removing one that three teams depend on is an outage.
This needs knowledge of who consumes what.
Job 4 — Should this deploy be blocked?
The organisational one.
Someone has to decide that a finding stops a merge, and someone has to be able to override it without turning the whole check off.
Most tools do one or two of these well.
Nobody does all four, including me.
Job 1: breaking-change detection — oasdiff wins oasdiff is open source, has several hundred breaking-change checks, ships a CLI and a GitHub Action, and has a hosted web diff where you paste two specs.
It's the reference implementation.
If your requirement is "tell me when a PR breaks the spec," install it this afternoon and skip the rest of this post.
I'll say the uncomfortable part plainly, since it's the honest framing: breaking-change detection is commoditised.
It's table stakes.
Nobody should pay for it as a standalone feature, mine included.
Job 2: linting — Spectral and vacuum Spectral is the de facto standard, and vacuum is a much faster Go implementation that reads the same ruleset format.
Both free.
Both good.
Stoplight, Apigee API hub and Azure API Center all bundle Spectral under the hood, so if you're on one of those you already have this.
The gap isn't the linter.
It's what happens to the output. prints a wall of warnings, someone puts it in CI, it fails on 200 findings the first day, and by week three it's .
I've watched this happen at two companies.
Job 3: consumer-aware verification — PactFlow, and this is where money lives Pact and PactFlow solve the real version.
Consumers publish what they actually use, providers publish what they offer, and a broker tells you whether a given pair is compatible. is the piece worth stealing conceptually: That's not "did the spec change." It's "will this specific build break a consumer that is live right now." PactFlow is the mature product here and starts around $99/month.
If you want full consumer-driven contract testing with the Pact DSL and runtime verification, they go deeper than anything spec-only can — including mine.
Spec-to-spec comparison can't tell you a consumer reads a field it never declared.
Runtime pact verification can.
Worth knowing: Optic was archived in January
2026.
If you're running it, it's unmaintained.
That surprised a few people I've spoken to.
Job 4: the gate — the part I think is underserved Every tool above produces findings.
Very few help you turn findings into a decision that survives contact with a deadline.
This is the gap I built SpecShield for, so weigh what follows accordingly.
A score, not a wall of warnings Governance findings get weighted by severity into a 0–100 score and an A–F grade: Any error-severity finding caps the grade at C, on the theory that an API with an unresolved error is not an "A" API no matter how many nice descriptions it has.
A threshold is something a team can agree on. "Zero warnings" is something a team agrees on once and then abandons.
Three rule packs ship built in: Pack Rules For 20 Design consistency — , descriptions, kebab-case paths, camelCase properties, array and string bounds 4 HTTPS-only servers, security field defined, no credentials in query strings, no HTTP Basic 3 The highest-signal rules only, for adding a gate to a legacy spec without drowning Waivers, so the gate survives This is the feature I'd argue matters most, and it's the least exciting to demo.
When a gate blocks something at 6pm on a Friday, the team has two options: fix it, or disable the check.
Everyone disables the check.
Then it never comes back on.
So findings can be waived at the org level, with the waiver recorded and revocable: The waiver applies to the verdict, the rule stays on for everyone else, and there's a record of who accepted what.
The gate stays credible because there's a pressure valve that isn't "delete the CI step." Rulesets that aren't a lock-in trap Custom rulesets are Spectral-format YAML.
There's a portability validator that actively rejects engine-specific constructs — vacuum-only , vendor rule-id prefixes, engine-specific functions.
That's deliberate.
A ruleset you write should run somewhere else if you leave.
Governance rules encode institutional knowledge, and holding that hostage is a bad way to keep customers.
Output CI already understands SARIF, JUnit, HTML and JSON.
SARIF matters most — GitHub ingests it natively, so findings show up as annotations on the diff instead of buried in log output nobody opens.
The deploy gate Consumers register the contract they depend on.
Before a deploy, you ask whether it's safe.
Same shape as PactFlow's, built on OpenAPI rather than the Pact DSL — lighter to adopt, less powerful at runtime.
That's a genuine trade, not a free win.
Where it runs The same engine is exposed through a CLI, a GitHub Action, a GitHub App that posts a check run and a sticky PR comment, an IntelliJ plugin, and an MCP server so coding agents can run the gate before opening a PR.
One thing worth stating clearly: it's fully deterministic.
No LLM anywhere in the analysis.
The same spec always produces the same score, and every deducted point traces to a named rule.
In a market currently bolting language models onto linters, "explainable and reproducible" felt like the more useful property for something that blocks deploys.
The comparison, honestly Breaking changes Linting Consumer-aware Deploy gate Waivers Cost oasdiff ✅ Best in class ❌ ❌ Partial ❌ Free Spectral / vacuum ❌ ✅ Standard ❌ ❌ ❌ Free PactFlow ✅ Partial ✅ Deepest ✅ Partial ~$99/mo+ Bump.sh ✅ Partial ❌ ❌ ❌ ~$50/mo+ Optic ✅ ✅ ❌ ❌ ❌ Archived SpecShield ✅ ✅ ✅ ✅ ✅ Free tier Reading that table fairly: oasdiff beats me at raw breaking-change detection.
More checks, longer track record, larger community.
PactFlow beats me at consumer-driven testing.
Runtime verification catches things spec comparison structurally cannot.
Spectral has a far bigger ruleset ecosystem than my three packs.
Apigee and Azure API Center aren't competitors at all — they're runtime API management.
Different layer, and complementary.
Where I think the combination earns its place: one gate that does breaking changes and governance, with a threshold and a waiver workflow, wired into the PR rather than a separate dashboard.
If you don't need the gate, the free tools are genuinely enough.
I'd rather say that than have you find out in week two.
What I'd actually recommend Just starting? oasdiff in CI.
Today.
Takes twenty minutes and catches the majority o