Feature-level @fail tag is not inherited by scenarios (v1 → v2 regression)
Description
Karate's @fail tag inverts a scenario's pass/fail result: a scenario that fails is reported as passed (used for negative reproductions that are expected to fail).
In v2, a @failtag declared at the Feature level does not propagate to the feature's scenarios. Only a @fail placed directly on each Scenario / Scenario Outline is honored. This is inconsistent with v1, where a feature-level `@fail ' applies to all scenarios in the feature.
NOTE:- feature-level @ignore does propagate to every scenario in v2.
Steps to Reproduce
@fail
Feature: match on null intermediate path incorrectly passes in v2
Scenario Outline: Case 1 - <scenario> - Examples expansion with preceding matches
* def response =
"""
{
"data": {
"a": {
"b": {
"range": {
"start": "2024-01-01",
"end": "2024-12-31"
},
"nullNode": null
}
}
}
}
"""
# Preceding assertions that pass
And match response.data.a.b.range.start == "#notnull"
And match response.data.a.b.range.end == "#notnull"
# nullNode is null — accessing child props should fail
And match response.data.a.b.nullNode.id == '<id>'
And match response.data.a.b.nullNode.name == '<name>'
Examples:
| scenario | id | name |
| string literals | ABC-123 | Some Value |
# --- Case 2: Inline string (no Examples) with preceding assertions ---
Scenario: Case 2 - Inline strings with preceding matches
* def response =
"""
{
"data": {
"a": {
"b": {
"range": {
"start": "2024-01-01",
"end": "2024-12-31"
},
"nullNode": null
}
}
}
}
"""
And match response.data.a.b.range.start == "#notnull"
And match response.data.a.b.range.end == "#notnull"
And match response.data.a.b.nullNode.id == 'ABC-123'
And match response.data.a.b.nullNode.name == 'Some Value'
# --- Case 3: No preceding assertions ---
Scenario Outline: Case 3 - <scenario> - No preceding matches
* def response =
"""
{
"data": {
"a": {
"nullNode": null
}
}
}
"""
And match response.data.a.nullNode.id == '<id>'
And match response.data.a.nullNode.name == '<name>'
Examples:
| scenario | id | name |
| simple null path | ABC-123 | Some Value |
# --- Case 4: Number comparison ---
Scenario Outline: Case 4 - <scenario> - Number comparison on null path
* def response =
"""
{
"data": {
"a": {
"nullNode": null
}
}
}
"""
And match response.data.a.nullNode.count == <count>
Examples:
| scenario | count |
| number match | 42 |
# --- Case 5: Boolean comparison ---
Scenario Outline: Case 5 - <scenario> - Boolean comparison on null path
* def response =
"""
{
"data": {
"a": {
"nullNode": null
}
}
}
"""
And match response.data.a.nullNode.active == <active>
Examples:
| scenario | active |
| boolean match | true |Expected Behavior
The feature-level @fail should apply to all 5 scenarios (as in v1 and as @ignore does). Every scenario fails, @fail inverts each result, and the feature reports as passed — getScenarioFailedCount() == 0.
Actual Behavior
The feature-level @fail is ignored. The 5 failing scenarios are reported as failures
Karate Version
2.1.2
Java Version
25.0.2
Operating System
macOS
Source: karatelabs/karate