waza

Waza

A Go CLI for evaluating AI agent skills — scaffold eval suites, run benchmarks, and compare results across models.

Getting Started / Docs

Installation

Binary Install (recommended)

Download and install the latest pre-built binary with the Bash install script on macOS, Linux, or Windows Bash environments such as Git Bash, MSYS2, or Cygwin:

curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash

The Bash script auto-detects the OS and architecture of the environment where Bash is running (linux/darwin/windows, amd64/arm64), downloads the latest standalone waza CLI release, verifies the checksum, and installs to /usr/local/bin (or ~/bin if not writable).

For native Windows PowerShell:

irm https://raw.githubusercontent.com/microsoft/waza/main/install.ps1 | iex

The PowerShell script downloads the latest standalone native Windows waza binary, verifies the checksum, and installs to an existing waza.exe location or %LOCALAPPDATA%\Microsoft\Waza. On Windows, piping the Bash command from PowerShell may invoke WSL and install the Linux binary inside WSL.

Or browse the GitHub Releases page and choose the standalone waza binary assets for the version you want.

Install from Source

Requires Go 1.26+:

Note: due to the use of LFS artifacts you cannot install waza using go install. To install waza outside of a normal release, clone the repository:

git clone https://github.com/microsoft/waza.git
cd waza

# ensure git LFS-based artifacts are available (for embedded copilot binaries)
git lfs install
git lfs pull

go build -o waza ./cmd/waza
./waza 

Waza bundles the GitHub Copilot CLI used by the copilot-sdk executor and extracts it to the local user cache on first use. Set COPILOT_CLI_PATH only when you need to force a specific Copilot CLI binary.

Azure Developer CLI (azd) Extension

Waza is also available as an azd extension:

# Add the waza extension registry
azd ext source add -n waza -t url -l https://raw.githubusercontent.com/microsoft/waza/main/registry.json

# Install the extension
azd ext install microsoft.azd.waza

# Verify it's working
azd waza --help

Once installed, all waza commands are available under azd waza. For example:

azd waza init my-eval --interactive
azd waza run examples/code-explainer/eval.yaml -v

Update Notifications

Waza automatically checks for new versions in the background. If an update is available, a notice appears after command output:

A newer version of waza is available: v0.24.0 → v0.28.0. Run: waza update

Run waza update to download and execute the official OS-specific installer after an explicit confirmation prompt. It uses the Bash installer on macOS/Linux and the PowerShell installer on native Windows. Use waza update --yes to skip the prompt in scripted environments. The check is non-blocking (never slows commands), cached for 24 hours, and can be disabled with --no-update-check or WAZA_NO_UPDATE_CHECK=1.

Quick Start

For New Users: Get Started in 5 Minutes

See Getting Started Guide for a complete walkthrough:

# Initialize a new project
waza init my-project && cd my-project

# Create a new skill
waza new skill my-skill

# Define the skill in skills/my-skill/SKILL.md
# Write evaluation tasks in evals/my-skill/tasks/
# Add test fixtures in evals/my-skill/fixtures/

# Run evaluations
waza run my-skill

# Check skill readiness
waza check my-skill

All Commands

…

Commands

waza update

Update waza to the latest release by running the official OS-specific installer after confirmation.

Example:

waza update
waza update --yes

waza init [directory]

Initialize a waza project workspace with separated skills/ and evals/ directories. Idempotent — creates only missing files.

Creates:

• skills/ — Skill definitions directory
• evals/ — Evaluation suites directory
• .github/workflows/eval.yml — CI/CD pipeline for running evals on PR
• .gitignore — Waza-specific exclusions
• README.md — Getting started guide for your project

Example:

waza init my-project
# Optionally creates first skill interactively

waza init my-project --no-skill
# Skip skill creation prompt

waza new skill

Create a new skill with scaffolded structure and evaluation suite. Detects workspace context and adapts output. In interactive mode, the wizard collects spec-aligned metadata: name, description, trigger phrases, and anti-trigger phrases.

Modes:

Project mode (detects skills/ directory):

project/
├── skills/{skill-name}/SKILL.md
└── evals/{skill-name}/
    ├── eval.yaml                 # or files.evalFile
    ├── tasks/*.yaml              # or files.taskGlob / files.taskFileSuffix
    └── fixtures/

APM-managed skills are detected from their compiled output without symlinks:

project/
├── skills/{skill-name}/apm.yml
├── skills/{skill-name}/.apm/skills/{skill-name}/SKILL.md
└── skills/{skill-name}/eval.yaml

When both skills/{skill-name}/SKILL.md and the APM compiled .apm/skills/{skill-name}/SKILL.md exist for the same skill, the top-level SKILL.md takes precedence.

Standalone mode (no skills/ detected):

{skill-name}/
├── SKILL.md
├── evals/
│   ├── eval.yaml                 # or files.evalFile
│   ├── tasks/*.yaml              # or files.taskGlob / files.taskFileSuffix
│   └── fixtures/
├── .github/workflows/eval.yml
├── .gitignore
└── README.md

Example:

# In project mode (explained Modes section, above): creates skills/code-explainer/SKILL.md + evals/code-explainer/
waza new skill code-explainer

# In standalone mode (explained Modes section, above): creates code-explainer/ self-contained directory
waza new skill code-explainer

waza new eval

Scaffold an eval suite from an existing SKILL.md (reads frontmatter trigger hints from USE FOR and DO NOT USE FOR).

Creates:

• evals//
• evals//tasks/positive-trigger-1
• evals//tasks/positive-trigger-2
• evals//tasks/negative-trigger-1

Generated eval and task filenames are configurable in .waza.yaml:

files:
  evalFile: waza-eval.yaml
  taskGlob: tasks/*.waza-task.yaml
  taskFileSuffix: .waza-task.yaml

Example:

# Default output location
waza new eval code-explainer

# Custom eval path
waza new eval code-explainer --output evals/custom-code-explainer/eval.yaml

waza new task from-prompt

Run a prompt through Copilot and generate a task YAML with inferred validators based on observed behavior (response text, tool usage, and invoked skills).

Example:

# Record a prompt and generate a reusable task YAML
waza new task from-prompt "Refactor this function for readability" evals/code-explainer/tasks/refactor-readability.yaml

# Add metadata and overwrite an existing file
waza new task from-prompt "Explain this diff and risks" evals/code-explainer/tasks/diff-analysis.yaml \
  --testname diff-analysis \
  --tags recorded,regression \
  --overwrite

waza run

Run an evaluation benchmark from a spec file.

**Result