scenarigo

A scenario-based API testing tool for HTTP/gRPC server.

Overview

Scenarigo is a scenario-based API testing tool for HTTP/gRPC server. It is written in Go and provides a plugin feature that enables you to extend by writing Go code. You can write test scenarios as YAML files and executes them.

title: get scenarigo repository
vars:
  user: scenarigo
  repo: scenarigo
steps:
- title: get repository
  protocol: http
  request:
    method: GET
    url: 'https://api.github.com/repos/{{vars.user}}/{{vars.repo}}'
  expect:
    code: OK
    body:
      id: '{{int($) > 0}}'
      name: '{{vars.repo}}'

Features

• Multi-Protocol Support - Test both HTTP/REST and gRPC APIs
• YAML-based Scenarios - Write test scenarios in a readable, declarative format
• Template Strings - Dynamic value generation and validation with template expressions
• Plugin System - Extend functionality by writing custom Go plugins
• Variables and Secrets - Manage test data with variable scoping and secret masking
• Retry Policies - Built-in retry with constant or exponential backoff strategies
• Conditional Execution - Control test flow with conditional step execution
• ytt Integration - Advanced templating and overlay capabilities for test scenarios

Quick Start

Installation

Install Scenarigo using Go:

$ go install github.com/scenarigo/scenarigo/cmd/scenarigo@latest

Your First Test

Create a simple test scenario file hello.yaml:

title: Hello Scenarigo
steps:
- title: Check GitHub API
  protocol: http
  request:
    method: GET
    url: https://api.github.com/repos/scenarigo/scenarigo
  expect:
    code: OK
    body:
      name: scenarigo

Running Tests

Create a configuration file and run the test:

# Initialize configuration
$ scenarigo config init

# Run the test
$ scenarigo run hello.yaml
ok      hello.yaml     0.123s

That's it! You've just run your first Scenarigo test. Continue reading to learn more advanced features.

Installation (Detailed)

go install command (recommend)

$ go install github.com/scenarigo/scenarigo/cmd/scenarigo@latest

from release page

Go to the releases page and download the zip file. Unpack the zip file, and put the binary to a directory in your $PATH.

You can download the latest command into the ./scenarigo directory with the following one-liner code. Place the binary ./scenarigo/scenarigo into your $PATH.

$ version=$(curl -s https://api.github.com/repos/scenarigo/scenarigo/releases/latest | jq -r '.tag_name') && \
    go_version=$(echo -n $(curl -s 'https://go.dev/VERSION?m=text' | head -n 1)) && \
    curl -sLJ https://github.com/scenarigo/scenarigo/releases/download/${version}/scenarigo_${version}_${go_version}_$(uname)_$(uname -m).tar.gz -o scenarigo.tar.gz && \
    mkdir ./scenarigo && tar -zxvf ./scenarigo.tar.gz -C ./scenarigo && rm scenarigo.tar.gz

Notes: If you use the plugin mechanism, the scenarigo command and plugins must be built using the same version of Go.

Setup

You can generate a configuration file scenarigo.yaml via the following command.

$ scenarigo config init

…

Usage

scenarigo run executes test scenarios based on the configuration file.

schemaVersion: config/v1

scenarios:
- github.yaml

title: get scenarigo repository
steps:
- title: GET https://api.github.com/repos/scenarigo/scenarigo
  vars:
    user: scenarigo
    repo: scenarigo
  protocol: http
  request:
    method: GET
    url: "https://api.github.com/repos/{{vars.user}}/{{vars.repo}}"
  expect:
    code: OK
    body:
      name: "{{vars.repo}}"

$ scenarigo run
ok      github.yaml     0.068s

Alternatively, provide the paths to specific test files as arguments.

$ scenarigo run github.yaml

You can see all commands and options by scenarigo help.

…

How to write test scenarios

You can write test scenarios easily in YAML. A test scenario consists of steps that are executed sequentially from top to bottom. Each step represents an API request (HTTP or gRPC) and its expected response.

Scenarigo supports testing both HTTP/REST and gRPC APIs. The following sections describe how to write tests for each protocol.

HTTP Testing

Send HTTP requests

This simple example has a step that sends a GET request to http://example.com/message.

title: check /message
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message

To send a query parameter, add it directly to the URL or use the query field.

title: check /message
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1

You can use other methods to send data to your APIs.

title: check /message
steps:
- title: POST /message
  protocol: http
  request:
    method: POST
    url: http://example.com/message
    body:
      message: hello

By default, Scenarigo will send body data as JSON. If you want to use other formats, set the Content-Type header.

title: check /message
steps:
- title: POST /message
  protocol: http
  request:
    method: POST
    url: http://example.com/message
    header:
      Content-Type: application/x-www-form-urlencoded
    body:
      message: hello

Available Content-Type header to encode request body is the following.

• application/json (default)
• text/plain
• application/x-www-form-urlencoded

Check HTTP responses

You can test your APIs by checking responses. If the result differs from the expected values, Scenarigo aborts the execution of the test scenario and notifies the error.

Scenarigo provides three ways to validate response values in the expect field:

1. Exact Matching - Compare values directly for equality
2. Template Expressions - Use conditional expressions with the actual value $
3. Assertion Functions - Use built-in assertion functions for common validations

Exact Matching

The simplest way to validate responses is to specify the expected values directly. Scenarigo will compare them for exact equality.

title: exact matching
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1
  expect:
    code: OK
    header:
      Content-Type: application/json; charset=utf-8
    body:
      id: 1
      message: hello

This method is best when you know the exact expected value and want a simple equality check.

Template Expressions

For more flexible validations, you can use template string expressions with the actual value represented by $. This allows you to write conditional expressions and perform calculations.

title: template expressions
steps:
- title: GET /message
  protocol: http
  request:
    method: GET
    url: http://example.com/message
    query:
      id: 1
  expect:
    code: OK
    header:
      Content-Type: application/json; charset=utf-8
    body:
      id: '{{int($) > 0}}'                                    # Check if id is positive
      message: '{{"hello" + " world"}}'                       # String concatenation
      timestamp: '{{time($) > time("2024-01-01T00:00:00Z")}}' # Time comparison

Template expressions are useful when:

• You need to perform range checks or comparisons
• The exact value is unknown but must satisfy certain conditions
• You want to perform type conversions before validation

Assertion Functions

Scenarigo provides the assert variable with built-in assertion functions for common validation patterns. These functions offer a more expressive and readable way to validate responses.

Available Assertion Functions:

  Function
  Usage
  Description






  any
  '{{assert.any}}'
  Always passes without validating the actual value




  notZero
  '{{assert.notZero}}'
  Ensures the value is not a zero value




  regexp
  '{{assert.regexp("^[a-z]+$")}}'
  Ensures the value matches the regular expression pattern




  length
  '{{assert.length(3)}}'
  Ensures the length of a string, array, slice, or map equals the expected value




  greaterThan
  '{{assert.greaterThan(10)}}'
  Ensures the value is greater than the expected value




  greaterThanOrEqual
  '{{assert.greaterThanOrEqual(10)}}'
  Ensures the value is greater than or equal to the expected value




  lessThan
  '{{assert.lessThan(100)}}'
  Ensures the value is less than the expected value




  lessThanOrEqual
  '{{assert.lessThanOrEqual(100)}}'
  Ensures the value is less than or equal to the expected value




  contains
  '{{assert.contains <-}}': value
  Ensures the array or slice contains the specified value (uses Left Arrow Function)




  notContains
  '{{assert.notContains <-}}': value
  Ensures the array or slice does not contain the specified value (uses Left Arrow Function)




  and
  '{{assert.and <-}}': [assertion1, assertion2]
  Ensures the value passes all assertions (uses Left Arrow Function)




  or
  '{{assert.or <-}}': [assertion1, assertion2]
  Ensures the value passes at least one of the assertions (uses Left Arrow Function)

Example:

title: assertion functions
steps:
- title: GET /users/1
  protocol: http
  request:
    method: GET
    url: http://example.com/users/1
  expect:
    code: OK
    body:
      id: '{{assert.notZero}}'
      name: '{{assert.regexp("^[A-Za-z ]+$")}}'
      age: '{{assert.greaterThanOrEqual(0)}}'
      email: '{{assert.regexp("^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$")}}'
      tags: '{{assert.length(3)}}'

For advanced assertions, you can combine multiple conditions using the and and or functions with the Left Arrow Function syntax:

expect:
  body:
    # Ensures age is between 20 and 65
    age:
      '{{assert.and  max {
			return fmt.Errorf("%d is not in range [%d, %d]", num, min, max)
		}
		return nil
	})
}

Usage in Test Scenarios:

title: custom assertions
plugins:
  myassert: myassert.so
steps:
- title: POST /users
  protocol: http
  request:
    method: POST
    url: http://example.com/users
    body:
      name: John Doe
      email: [email protected]
      age: 30
  expect:
    code: Created
    body:
      email: '{{plugins.myassert.EmailFormat}}'
      age: '{{plugins.myassert.InRange(18, 65)}}'

The assert.Assertion interface requires only one method:

type Assertion interface {
    Assert(v any) error
}

You can implement this interface directly or use the convenient assert.AssertionFunc adapter to convert a function into an assertion.

Combining Validation Methods

You can combine all three validation methods in a single test scenario to leverage the strengths of each approach:

…

Custom Client

Scenarigo allows you to use custom clients defined in plugins. You can pass custom clients through the client field in your test scenarios.

For HTTP tests, you can pass a custom *http.Client instance defined in your plugin:

package main

import (
	"net/http"
	"time"
)

var CustomHTTPClient = &http.Client{
	Timeout: 10 * time.Second,
	// Add custom transport, middleware, etc.
}

title: test with custom HTTP client
plugins:
  client: client.so
steps:
- title: GET /api/resource
  protocol: http
  request: