---
title: "microsoft/waza"
description: "CLI / Framework for Agent Skills - create, test, measure and improve skill quality and effectiveness"
source: https://github.com/microsoft/waza
ref: main
license: MIT
licenseName: "MIT License"
canonical: https://skillsdocs.com/microsoft/waza
base: https://github.com/microsoft/waza/blob/main/
provenance: mixed
chapters: 17
inlined: 17
withheld: 0
words: 6955
updated: 2026-09-24T16:49:08Z
generator: "Skills Docs"
---

> **microsoft/waza** — every Agent Skill in this repository, inlined verbatim.
>
> Canonical HTML: https://skillsdocs.com/microsoft/waza
> Per-skill Markdown: https://skillsdocs.com/microsoft/waza/<skill>.md
> Machine manifest: https://skillsdocs.com/microsoft/waza/.well-known/agent-skills/index.json
> JSON: https://skillsdocs.com/api/v1/books/microsoft/waza
> Install: `npx skills add microsoft/waza`
> Upstream: https://github.com/microsoft/waza @ `main`
> Licence: MIT
>
> Content is mirrored from GitHub and © its authors, served unmodified. Takedown: https://github.com/DreambaseAI/skillsdocs/issues/new?labels=takedown&title=Takedown+request

# microsoft/waza

CLI / Framework for Agent Skills - create, test, measure and improve skill quality and effectiveness

- **Skills:** 17
- **Authorship:** mixed — 1 of 17 are credited — skills in use here, not published from here
- **Inlined:** 17 (licence detected)
- **Words:** 6,955
- **Reading time:** 33 min
- **Stars:** 1,323

## Table of contents

1. [agent-collaboration](https://skillsdocs.com/microsoft/waza/agent-collaboration.md) — Standard collaboration patterns for all squad agents — worktree awareness, decisions, cross-agent communication
2. [error-recovery](https://skillsdocs.com/microsoft/waza/error-recovery.md) — Standard recovery patterns for all squad agents. When something fails, adapt — don't just report the failure.
3. [git-workflow](https://skillsdocs.com/microsoft/waza/git-workflow.md) — Squad branching model: dev-first workflow with insiders preview channel
4. [reviewer-protocol](https://skillsdocs.com/microsoft/waza/reviewer-protocol.md) — Reviewer rejection workflow and strict lockout semantics
5. [secret-handling](https://skillsdocs.com/microsoft/waza/secret-handling.md) — Never read .env files or write secrets to .squad/ committed files
6. [session-recovery](https://skillsdocs.com/microsoft/waza/session-recovery.md) — Find and resume interrupted Copilot CLI sessions using session_store queries
7. [squad-commands](https://skillsdocs.com/microsoft/waza/squad-commands.md) — Categorized catalog of common Squad operations. Coordinator reads this file and presents it as an interactive menu when the user asks for available commands or…
8. [squad-conventions](https://skillsdocs.com/microsoft/waza/squad-conventions.md) — Core conventions and patterns used in the Squad codebase
9. [squad-version-check](https://skillsdocs.com/microsoft/waza/squad-version-check.md) — No description.
10. [test-discipline](https://skillsdocs.com/microsoft/waza/test-discipline.md) — Update tests when changing APIs — no exceptions
11. [{skill-name}](https://skillsdocs.com/microsoft/waza/squad-templates.md) — {what this skill teaches agents}
12. [{skill-name}](https://skillsdocs.com/microsoft/waza/templates.md) — {what this skill teaches agents}
13. [code-explainer](https://skillsdocs.com/microsoft/waza/code-explainer.md) — Explains code snippets in plain English, breaking down what the code does step by step. Perfect for learning, code reviews, or documentation.
14. [waza-interactive](https://skillsdocs.com/microsoft/waza/waza-interactive.md) — Interactive workflow partner for creating, testing, and improving AI agent skills with waza. USE FOR: run my evals, check my skill, compare models, create eval…
15. [waza](https://skillsdocs.com/microsoft/waza/waza.md) — **WORKFLOW SKILL** - Evaluate AI agent skills using structured benchmarks with YAML specs, fixture isolation, and pluggable validators. USE FOR: run waza, waza…
16. [waza-runner](https://skillsdocs.com/microsoft/waza/waza-runner.md) — Run evaluations on Agent Skills to measure their effectiveness. USE FOR: "run skill evals", "evaluate my skill", "test skill quality", "check skill triggers",…
17. [azd-publish](https://skillsdocs.com/microsoft/waza/azd-publish.md) — Prepare and publish a new version of the waza azd extension. USE FOR: "publish extension", "release new version", "bump version", "prepare release", "update ch…


## Front matter

_The repository README, verbatim except that relative links are resolved against https://github.com/microsoft/waza/blob/main/._

# Waza

A Go CLI for evaluating AI agent skills — scaffold eval suites, run benchmarks, and compare results across models.

📖 **[Getting Started / Docs](https://microsoft.github.io/waza/)**

## Installation

### Binary Install (recommended)

Download and install the latest pre-built binary with the Bash install script on macOS, Linux, or Windows Bash environments such as Git Bash, MSYS2, or Cygwin:

```bash
curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash
```

The Bash script auto-detects the OS and architecture of the environment where Bash is running (linux/darwin/windows, amd64/arm64), downloads the latest standalone `waza` CLI release, verifies the checksum, and installs to `/usr/local/bin` (or `~/bin` if not writable).

For native Windows PowerShell:

```powershell
irm https://raw.githubusercontent.com/microsoft/waza/main/install.ps1 | iex
```

The PowerShell script downloads the latest standalone native Windows `waza` binary, verifies the checksum, and installs to an existing `waza.exe` location or `%LOCALAPPDATA%\Microsoft\Waza`. On Windows, piping the Bash command from PowerShell may invoke WSL and install the Linux binary inside WSL.

Or browse the [GitHub Releases](https://github.com/microsoft/waza/releases) page and choose the standalone waza binary assets for the version you want.

### Install from Source

Requires Go 1.26+:

Note: due to the use of LFS artifacts you cannot install waza using `go install`. To install waza outside of a normal release, clone the repository:

```bash
git clone https://github.com/microsoft/waza.git
cd waza

# ensure git LFS-based artifacts are available (for embedded copilot binaries)
git lfs install
git lfs pull

go build -o waza ./cmd/waza
./waza <waza command line>
```

Waza bundles the GitHub Copilot CLI used by the `copilot-sdk` executor and extracts it to the local user cache on first use. Set `COPILOT_CLI_PATH` only when you need to force a specific Copilot CLI binary.

### Azure Developer CLI (azd) Extension

Waza is also available as an [azd extension](https://learn.microsoft.com/azure/developer/azure-developer-cli/extensions/overview):

```bash
# Add the waza extension registry
azd ext source add -n waza -t url -l https://raw.githubusercontent.com/microsoft/waza/main/registry.json

# Install the extension
azd ext install microsoft.azd.waza

# Verify it's working
azd waza --help
```

Once installed, all waza commands are available under `azd waza`. For example:

```bash
azd waza init my-eval --interactive
azd waza run examples/code-explainer/eval.yaml -v
```

## Update Notifications

Waza automatically checks for new versions in the background. If an update is available, a notice appears after command output:

```
A newer version of waza is available: v0.24.0 → v0.28.0. Run: waza update
```

Run `waza update` to download and execute the official OS-specific installer after an explicit confirmation prompt. It uses the Bash installer on macOS/Linux and the PowerShell installer on native Windows. Use `waza update --yes` to skip the prompt in scripted environments. The check is non-blocking (never slows commands), cached for 24 hours, and can be disabled with `--no-update-check` or `WAZA_NO_UPDATE_CHECK=1`.

## Quick Start

### For New Users: Get Started in 5 Minutes

See **[Getting Started Guide](https://github.com/microsoft/waza/blob/main/docs/GETTING-STARTED.md)** for a complete walkthrough:

```bash
# Initialize a new project
waza init my-project && cd my-project

# Create a new skill
waza new skill my-skill

# Define the skill in skills/my-skill/SKILL.md
# Write evaluation tasks in evals/my-skill/tasks/
# Add test fixtures in evals/my-skill/fixtures/

# Run evaluations
waza run my-skill

# Check skill readiness
waza check my-skill
```

### All Commands

```bash
# Build
make build

# Initialize a project workspace
waza init [directory]

# Update waza to the latest release
waza update

# Create a new skill
waza new skill skill-name

# Create a new eval scaffold from an existing SKILL.md
waza new eval skill-name

# Generate a task YAML by recording a prompt run
waza new task from-prompt "Explain this code and suggest fixes" evals/code-explainer/tasks/recorded-task.yaml

# Check if a skill is ready for submission
waza check skills/my-skill

# Suggest an eval suite from SKILL.md
waza suggest skills/my-skill --dry-run
waza suggest skills/my-skill --apply

# Discover shared registry graders and add one to an eval
waza registry search factual --kind grader
waza registry add github.com/waza-evals/fact#factuality@v1.0.0 --eval eval.yaml --name factuality

# Verify eval coverage against SKILL.md requirements
waza spec verify skills/my-skill evals/my-skill/eval.yaml
waza spec verify skills/my-skill evals/my-skill/eval.yaml --fail --format github-actions

# Resolve remote grader refs and write waza.lock
waza get evals/my-skill/eval.yaml

# Note: 'generate' is available as an alias for 'new' (see below for new command)
# Note: Custom agents (.agent.md) are supported — see https://microsoft.github.io/waza/guides/custom-agents/

# Run evaluations (works with both skills and custom agents)
waza run examples/code-explainer/eval.yaml --context-dir examples/code-explainer/fixtures -v

# Grade output from a previous `waza run --output results.json ...`
waza grade eval.yaml --results results.json

# Compare results across models
waza compare results-gpt4.json results-sonnet.json

# Capture snapshots during a run and replay them later for determinism checks
waza run eval.yaml --snapshot ./snapshots/
waza replay ./snapshots/my-task-run1.json

# Run offline adversarial / fault-injection packs against a skill
waza adversarial --list-packs
waza adversarial --skill ./skills/my-skill --model gpt-4o

# Check whether a schema artifact needs migration
waza migrate eval.yaml

# Generate eval coverage grid
waza coverage --format markdown

# Count tokens in skill files
waza tokens count skills/

# Compare skill token budgets vs main
waza tokens compare main --skills --threshold 10

# Suggest token optimizations
waza tokens suggest skills/
```

## Commands

### `waza update`

Update waza to the latest release by running the official OS-specific installer after confirmation.

| Flag | Description |
|------|-------------|
| `--yes`, `-y` | Skip the confirmation prompt |

**Example:**
```bash
waza update
waza update --yes
```

### `waza init [directory]`

Initialize a waza project workspace with separated `skills/` and `evals/` directories. Idempotent — creates only missing files.

| Flag | Description |
|------|-------------|
| `--no-skill` | Skip the first-skill creation prompt |

Creates:
- `skills/` — Skill definitions directory
- `evals/` — Evaluation suites directory
- `.github/workflows/eval.yml` — CI/CD pipeline for running evals on PR
- `.gitignore` — Waza-specific exclusions
- `README.md` — Getting started guide for your project

**Example:**
```bash
waza init my-project
# Optionally creates first skill interactively

waza init my-project --no-skill
# Skip skill creation prompt
```

### `waza new skill <skill-name>`

Create a new skill with scaffolded structure and evaluation suite. Detects workspace context and adapts output. In interactive mode, the wizard collects spec-aligned metadata: name, description, trigger phrases, and anti-trigger phrases.

| Flag | Short | Description |
|------|-------|-------------|
| `--template` | `-t` | Template pack (coming soon) |

**Modes:**

*Project mode* (detects `skills/` directory):
```
project/
├── skills/{skill-name}/SKILL.md
└── evals/{skill-name}/
    ├── eval.yaml                 # or files.evalFile
    ├── tasks/*.yaml              # or files.taskGlob / files.taskFileSuffix
    └── fixtures/
```

*APM-managed skills* are detected from their compiled output without symlinks:
```
project/
├── skills/{skill-name}/apm.yml
├── skills/{skill-name}/.apm/skills/{skill-name}/SKILL.md
└── skills/{skill-name}/eval.yaml
```

When both `skills/{skill-name}/SKILL.md` and the APM compiled
`.apm/skills/{skill-name}/SKILL.md` exist for the same skill, the top-level
`SKILL.md` takes precedence.

*Standalone mode* (no `skills/` detected):
```
{skill-name}/
├── SKILL.md
├── evals/
│   ├── eval.yaml                 # or files.evalFile
│   ├── tasks/*.yaml              # or files.taskGlob / files.taskFileSuffix
│   └── fixtures/
├── .github/workflows/eval.yml
├── .gitignore
└── README.md
```

**Example:**
```bash
# In project mode (explained Modes section, above): creates skills/code-explainer/SKILL.md + evals/code-explainer/
waza new skill code-explainer

# In standalone mode (explained Modes section, above): creates code-explainer/ self-contained directory
waza new skill code-explainer
```

### `waza new eval <skill-name>`

Scaffold an eval suite from an existing `SKILL.md` (reads frontmatter trigger hints from `USE FOR` and `DO NOT USE FOR`).

Creates:
- `evals/<skill-name>/<files.evalFile>`
- `evals/<skill-name>/tasks/positive-trigger-1<files.taskFileSuffix>`
- `evals/<skill-name>/tasks/positive-trigger-2<files.taskFileSuffix>`
- `evals/<skill-name>/tasks/negative-trigger-1<files.taskFileSuffix>`

| Flag | Description |
|------|-------------|
| `--output <path>` | Custom path for the eval file (tasks are generated under sibling `tasks/`) |

Generated eval and task filenames are configurable in `.waza.yaml`:

```yaml
files:
  evalFile: waza-eval.yaml
  taskGlob: tasks/*.waza-task.yaml
  taskFileSuffix: .waza-task.yaml
```

**Example:**
```bash
# Default output location
waza new eval code-explainer

# Custom eval path
waza new eval code-explainer --output evals/custom-code-explainer/eval.yaml
```

### `waza new task from-prompt <prompt> <task-path>`

Run a prompt through Copilot and generate a task YAML with inferred validators based on observed behavior (response text, tool usage, and invoked skills).

| Flag | Description |
|------|-------------|
| `--model <name>` | Copilot model to run for recording (default: `claude-sonnet-4.5`) |
| `--testname <name>` | Test name and ID written into the generated task (default: `auto-generated-test`) |
| `--tags <a,b,...>` | Comma-separated tags to attach to the generated task |
| `--timeout <duration>` | Max time for prompt execution (default: `5m`) |
| `--overwrite` | Overwrite the output task file if it already exists |
| `--root <dir>` | Root directory used for skill discovery (default: `.`) |

**Example:**
```bash
# Record a prompt and generate a reusable task YAML
waza new task from-prompt "Refactor this function for readability" evals/code-explainer/tasks/refactor-readability.yaml

# Add metadata and overwrite an existing file
waza new task from-prompt "Explain this diff and risks" evals/code-explainer/tasks/diff-analysis.yaml \
  --testname diff-analysis \
  --tags recorded,regression \
  --overwrite
```

### `waza run <eval.yaml>`

Run an evaluation benchmark from a spec file.

| Flag | Short | Description |
|------|-------|-------------|
| `--context-dir <dir>` | | Fixture directory (default: `./fixtures` relative to spec) |
| `--output <file>` | `-o` | Save results to JSON |
| `--output-dir <dir>` | | Directory for structured output; each run creates a UTC-timestamped subdirectory of `<dir>`. Mutually exclusive with `--output`. |
| `--verbose` | `-v` | Detailed progress output |
| `--transcript-dir <dir>` | | Save per-task transcript JSON files |
| `--task <glob>` | | Filter tasks by name/ID pattern (repeatable) |
| `--parallel` | | Run tasks concurrently |
| `--workers <n>` | | Concurrent workers (default: auto, requires `--parallel`) |
| `--trials <n>` | | Run each task `n` times to detect flakiness (omit to use `config.trials_per_task`; if provided, `n` must be >= 1) |
| `--interpret` | | Print plain-language result interpretation |
| `--format <fmt>` | | Output format: `default` or `github-comment` (default: `default`) |
| `--cache` | | Enable result caching to speed up repeated runs |
| `--no-cache` | | Explicitly disable result caching |
| `--cache-dir <dir>` | | Cache directory (default: `.waza-cache`) |
| `--reporter <spec>` | | Output reporters: `json` (default), `junit:<path>` (repeatable) |
| `--baseline` | | A/B testing mode — runs each task twice (without skill = baseline, with skill = normal) and computes improvement scores |
| `--discover` | | Auto skill discovery — walks directory tree for SKILL.md + eval.yaml (root/tests/evals) |
| `--strict` | | Fail if any SKILL.md lacks eval coverage (use with `--discover`) |
| `--suggest` | | Generate a Copilot suggestion report based on test outcomes (`mock` engine emits a deterministic fake report) |
| `--output-dir <dir>` | | Directory for structured output; each run creates a UTC timestamped subdirectory. Mutually exclusive with `--output`. |
| `--tags <patterns>` | | Filter tasks by tags, using glob patterns (repeatable) |
| `--model <name>` | | Override model (repeatable for multi-model comparison) |
| `--recommend` | | Generate heuristic recommendation after multi-model run |
| `--judge-model <model>` | | Model for LLM-as-judge graders (overrides execution model) |
| `--session-log` | | Enable session event logging (NDJSON) |
| `--session-dir <dir>` | | Directory for session log files (default: current directory) |
| `--no-summary` | | Skip writing combined summary.json for multi-skill runs |
| `--update-snapshots` | | Update or create diff grader snapshot files to match current output |
| `--skip-graders` | | Skip grading (execution only); grade later with `waza grade` |
| `--keep-workspace` | | Preserve temp workspaces after execution for debugging |
| `--auto-file-issue` | | Auto-file or update a GitHub issue for failing runs (requires `gh` and `GITHUB_REPOSITORY`) |
| `--otel-exporter` | | Export OpenTelemetry traces using `otlp`, `stdout`, or `file`. Off by default. See [OpenTelemetry Tracing](https://microsoft.github.io/waza/guides/otel/). |
| `--otel-endpoint` | | OTLP endpoint (host:port or URL); only used with `--otel-exporter=otlp` |
| `--otel-headers` | | Comma-separated `key=value` OTLP headers (e.g. for auth) |
| `--otel-file` | | File path for span JSON when `--otel-exporter=file` |
| `--otel-include-payloads` | | Include prompt/tool-arg/tool-result/completion content in spans (default: redacted to `sha256`+length) |
| `--snapshot <dir>` | | Capture self-contained `snapshot.json` per task for later [`waza replay`](#waza-replay-snapshotjson). |
| `--snapshot-env-allow <patterns>` | | Allow-list of env var name patterns embedded in snapshots (default-deny; supports `WAZA_*` wildcards). |
| `--redact <path>` | | YAML redaction policy applied to snapshot output (merged with built-in defaults). |

**Result Caching**

Enable caching with `--cache` to store test results and skip re-execution on repeated runs:

```bash
# First run executes all tests and caches results
waza run eval.yaml --cache

# Second run uses cached results (much faster)
waza run eval.yaml --cache

# Clear the cache when needed
waza cache clear
```

Cached results are automatically invalidated when:
- Spec configuration changes (model, timeout, graders, etc.)
- Task definitions change
- Fixture files change

**Note:** Caching is automatically disabled for evaluations using non-deterministic graders (`behavior`, `prompt`).

### `waza get [eval.yaml | ref]`

Resolve remote grader refs and write `waza.lock`. When passed an eval file, `waza get` resolves every `graders[].ref`, downloads module contents into `~/.waza/cache/{host}/{org}/{repo}/{sha}/`, and pins each ref to a commit SHA and `sha256:` content digest. `waza run` requires a valid lock and cache entry for remote refs; it does not silently resolve unlocked refs during a run.

```bash
waza get eval.yaml
waza get github.com/waza-evals/fact#factuality@v1.0.0
```

**Exit Codes**

The `run` command uses exit codes to enable CI/CD integration:

| Exit Code | Condition | Description |
|-----------|-----------|-------------|
| `0` | Success | All tests passed |
| `1` | Test failure | One or more tests failed validation |
| `2` | Configuration error | Invalid spec, missing files, or runtime error |

Example CI usage:

```bash
# Fail the build if any tests fail
waza run eval.yaml || exit $?

# Capture specific exit codes
waza run eval.yaml
EXIT_CODE=$?
if [ $EXIT_CODE -eq 1 ]; then
  echo "Tests failed - check results"
elif [ $EXIT_CODE -eq 2 ]; then
  echo "Configuration error"
fi

# Post results as PR comment (GitHub Actions)
waza run eval.yaml --format github-comment > comment.md
gh pr comment $PR_NUMBER --body-file comment.md

# Generate JUnit XML for CI test reporting
waza run eval.yaml --reporter junit:results.xml

# Both JSON output and JUnit XML
waza run eval.yaml -o results.json --reporter junit:results.xml
```

**Note:** `waza generate` is an alias for `waza new`. Both commands support the same functionality with the `--output-dir` flag for specifying custom output locations.

### `waza compare <file1> <file2> [files...]`

Compare results from multiple evaluation runs side by side — per-task score deltas, pass rate differences, and aggregate statistics.

| Flag | Short | Description |
|------|-------|-------------|
| `--format <fmt>` | `-f` | Output format: `table` or `json` (default: `table`) |

### `waza replay <snapshot.json>`

Replay a task snapshot to verify deterministic reproduction. Snapshots are produced by `waza run --snapshot <dir>` and capture the prompt, fixture digests, ordered tool events, environment allow-list, and redacted grader outcomes.

```bash
# Capture during a run
waza run eval.yaml --snapshot ./snapshots/

# Re-check internal consistency (offline, fast)
waza replay ./snapshots/my-task-run1.json

# Bisect two snapshots and find first divergent turn
waza replay ./snapshots/a.json --bisect ./snapshots/b.json --json
```

| Flag | Description |
|------|-------------|
| `--mode <mode>` | Replay mode: `model-replay` (default, offline consistency check) or `live` (planned) |
| `--bisect <file>` | Path to second snapshot to bisect against the primary |
| `--json` | Emit machine-readable JSON instead of human summary |
| `--strict` | Re-check final status and grader outcome consistency (default true) |

Exit codes: `0` match, `1` divergence, `2` load/parse error.

### `waza adversarial`

Run offline adversarial / fault-injection packs against a skill. Two built-in packs ship with the binary: **prompt-injection** (probes resistance to indirect prompt injection via fixture files) and **scope-bypass** (probes refusal of out-of-scope actions like sending email, deleting files, or installing packages). Every task is `golden: true`, so unsafe outcomes also flip `waza gate` to exit 2.

```bash
# List built-in packs
waza adversarial --list-packs

# Run every pack against a skill
waza adversarial --skill ./skills/code-review --model gpt-4o

# Read pack selection from eval.yaml (schema 1.2 adversarial: block)
waza adversarial --spec eval.yaml --output adversarial.json

# Non-blocking CI smoke
waza adversarial --packs prompt-injection --on-unsafe-outcome warn
```

| Flag | Description |
|------|-------------|
| `--packs` | Comma-separated pack names (default: every built-in pack) |
| `--list-packs` | Print the pack catalog and exit |
| `--spec` | Inherit `adversarial:` block from an `eval.yaml` |
| `--on-unsafe-outcome` | `fail` (exit 2, default) or `warn` (exit 0) |
| `--engine`, `--skill`, `--model` | Forwarded to the underlying engine |
| `--output` | Write the full `results.json` to a file |

Exit codes: `0` all packs PASSED, `2` unsafe outcome with policy=fail (matches `waza gate`), `3` config error. See the [Adversarial harness guide](https://microsoft.github.io/waza/guides/adversarial/) for details.

### `waza migrate <file>`

Check a public schema artifact and migrate it to the current schema version when a future major schema requires it. The current schema is `1.2`, so v1 `eval.yaml` and `results.json` files are already current and no file changes are made.

```bash
waza migrate eval.yaml
waza migrate results.json
```

### `waza coverage [root]`

Generate a skill-to-eval coverage grid showing which skills are fully covered, partially covered, or missing evals.

**Note**: Full coverage requires tasks (via `tasks:` or `tasks_from:`) and 2+ grader types. The coverage percentage reflects only fully covered skills.

| Flag | Short | Description |
|------|-------|-------------|
| `--format <fmt>` | `-f` | Output format: `text`, `markdown`, or `json` (default: `text`) |
| `--path <dir>` | | Additional directory to scan for skills/evals (repeatable) |

### `waza spec verify [skill-path] [eval.yaml]`

Verify that an eval suite exercises the promises made in `SKILL.md`. The command deterministically parses the description, `USE FOR` triggers, `DO NOT USE FOR` triggers, and parameter blocks into requirement IDs such as `req-use-001` and `req-dont-001`, then maps each requirement to matching task IDs.

| Flag | Description |
|------|-------------|
| `--skill <path>` | Path to `SKILL.md` or a skill directory |
| `--eval <path>` | Path to `eval.yaml` |
| `--format <fmt>` | Output format: `human`, `json`, or `github-actions` |
| `--warn` | Report uncovered requirements and exit 0 (default); set false to suppress GitHub Actions warning annotations |
| `--fail` | Exit 1 when uncovered requirements are greater than or equal to `--threshold` |
| `--threshold <n>` | Uncovered requirement threshold for `--fail` (default: 1) |
| `--semantic` | Opt in to LLM-assisted semantic matching after deterministic matching |
| `--judge-model <model>` | Judge model for `--semantic` (defaults to `config.judge_model`, then `config.model`) |

**Example:**

```bash
waza spec verify skills/pr-summarizer evals/pr-summarizer/eval.yaml
waza spec verify --skill skills/pr-summarizer --eval evals/pr-summarizer/eval.yaml --format json
```

### `waza models`

List models available for evaluation via the Copilot SDK. Shows model IDs and metadata that can be used with `--model` flags in `waza run`, `waza quality`, and other commands.

Requires authentication via `copilot login`. Custom provider configuration only applies when creating or resuming Copilot SDK sessions.

| Flag | Description |
|------|-------------|
| `--json` | Output as JSON |

**Examples:**

```bash
# List available models in table format
waza models

# Output available models as JSON
waza models --json
```

### `waza cache clear`

Clear all cached evaluation results to force re-execution on the next run.

| Flag | Description |
|------|-------------|
| `--cache-dir <dir>` | Cache directory to clear (default: `.waza-cache`) |

### `waza registry search <query>`

Search configured registry indexes for reusable graders, eval bundles, and datasets. The default public registry source is `https://github.com/waza-evals`; project-level `.waza.yaml` can override sources with a top-level `registries:` list.

Registry search currently returns bundled sample metadata while live index integration is pending.

| Flag | Description |
|------|-------------|
| `--kind <kind>` | Filter by `grader`, `eval`, or `dataset` |
| `--registry <name>` | Search only the named registry source |
| `--format <format>` | Output `table` or `json` (default: `table`) |

### `waza registry add <ref>`

Append a remote grader preset reference to `eval.yaml`, resolve it with the same remote grader resolver as `waza get`, and update `waza.lock` with the pinned commit SHA and `sha256:` content digest.

| Flag | Description |
|------|-------------|
| `--eval <path>` | Eval file to update (default: `eval.yaml`) |
| `--name <alias>` | Local alias for the grader |
| `--set key=value` | Add a local override, repeatable (for example, `--set config.threshold=0.9`) |
| `--allow-exec` | Allow remote program graders without interactive confirmation |

### `waza dev [skill-path]`

Iteratively score and improve skill frontmatter in a SKILL.md file.

Use `--copilot` for a non-interactive, single-pass markdown report that:
1. Summarizes current skill details and token usage
2. Loads trigger test prompts as examples (when `trigger_tests.yaml` exists)
3. Requests Copilot suggestions for improving skill selection
4. Prints the report to stdout without applying any changes

When `--copilot` is set, iterative mode flags (`--target`, `--max-iterations`, `--auto`) are invalid.

| Flag | Description |
|------|-------------|
| `--target <level>` | Target adherence level for iterative mode: `low`, `medium`, `medium-high`, `high` (default: `medium-high`) |
| `--max-iterations <n>` | Maximum improvement iterations for iterative mode (default: 5) |
| `--auto` | Apply improvements without prompting in iterative mode |
| `--copilot` | Generate a non-interactive markdown report with Copilot suggestions |
| `--model <id>` | Model to use with `--copilot` |

### `waza check [skill-path]`

Check if a skill is ready for submission with a comprehensive readiness report.

Performs five types of checks:
1. **Compliance scoring** — Validates frontmatter adherence (Low/Medium/Medium-High/High)
2. **Token budget** — Checks if SKILL.md is within token limits (configurable in `.waza.yaml` `tokens.limits`)
3. **Evaluation suite** — Checks for the presence of eval.yaml
4. **Spec compliance** — Validates the skill against the agentskills.io spec (frontmatter structure, required fields, naming rules, directory match, description length, compatibility, license, and version)
5. **Advisory checks** — Detects quality and maintainability issues (reference module count, complexity classification, negative delta risk patterns, procedural content, and over-specificity)

Provides a plain-language summary and actionable next steps to improve the skill.

**Example output:**
```
🔍 Skill Readiness Check
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Skill: code-explainer

📋 Compliance Score: High
   ✅ Excellent! Your skill meets all compliance requirements.

📊 Token Budget: 450 / 500 tokens
   ✅ Within budget (50 tokens remaining).

🧪 Evaluation Suite: Found
   ✅ eval.yaml detected. Run 'waza run eval.yaml' to test.

📐 Spec Compliance (agentskills.io)
   ✅ spec-frontmatter    Frontmatter structure valid with required fields
   ✅ spec-allowed-fields All frontmatter fields are spec-allowed
   ✅ spec-name           Name follows spec naming rules
   ✅ spec-dir-match      Directory name matches skill name
   ✅ spec-description    Description is valid
   ✅ spec-license        License field present
   ✅ spec-version        metadata.version present

🔬 Advisory Checks
   ✅ module-count        Found 2 reference modules (2-3 is optimal)
   ✅ complexity          Complexity: detailed (350 tokens, 2 modules)
   ✅ negative-delta-risk No negative delta risk patterns detected
   ✅ procedural-content  Description contains procedural language
   ✅ over-specificity    No over-specificity patterns detected

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📈 Overall Readiness
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

✅ Your skill is ready for submission!

🎯 Next Steps
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

✨ No action needed! Your skill looks great.

Consider:
  • Running 'waza run eval.yaml' to verify functionality
  • Sharing your skill with the community
```

**Usage:**
```bash
# Check current directory
waza check

# Check specific skill
waza check skills/my-skill

# Suggested workflow
waza check skills/my-skill     # Check readiness
waza dev skills/my-skill       # Improve compliance if needed
waza check skills/my-skill     # Verify improvements
```

### `waza quality <skill-path>`

Use an LLM-as-Judge to evaluate skill content quality across five dimensions:
clarity, completeness, trigger precision, scope coverage, and anti-patterns.

| Flag | Description |
|------|-------------|
| `--model <model>` | Model to use as judge (default: project default model) |
| `--format table\|json` | Output format (default: `table`) |
| `--rubric <path>` | Path to custom rubric file (reserved for future use) |

**Examples:**
```bash
# Evaluate skill quality (table output)
waza quality skills/code-explainer

# JSON output for CI integration
waza quality skills/code-explainer --format json

# Use a specific model as judge
waza quality skills/code-explainer --model gpt-4o
```

### `waza suggest <skill-path>`

Use an LLM to analyze `SKILL.md` and generate suggested evaluation artifacts.

| Flag | Description |
|------|-------------|
| `--model <model>` | Model to use for suggestions (default: project default model) |
| `--dry-run` | Print suggested output to stdout (default) |
| `--apply` | Write files to disk |
| `--force` | Allow `--apply` to overwrite existing eval/task/fixture files (requires `--apply`) |
| `--count <n>` | Generate exactly N tasks (default: at least 3 + 1 negative) |
| `--focus <category>` | Steer generation toward one of `triggers`, `negative-triggers`, `edge-fixtures`, `do-not-use-for`, `parameters` |
| `--output-dir <dir>` | Output directory (default: `<skill-path>/evals`) |
| `--format yaml\|json` | Output format (default: `yaml`) |

Each generated task entry carries a `confidence` score in `[0, 1]` and a `rationale` string pointing to the SKILL.md span it was derived from. Both appear in dry-run output but are kept outside the written task YAML (the task schema rejects unknown fields).

`--apply` is merge-safe: an existing `eval.yaml` is never overwritten (new task files are picked up by its existing `tasks:` glob), and existing task files (by path or by task `id`) cause `--apply` to fail with a diff unless `--force` is also passed. Existing fixture files are also preserved unless `--force` is set.

**Examples:**
```bash
# Preview generated eval/task/fixture files as YAML
waza suggest skills/code-explainer --dry-run

# Generate exactly 5 negative-trigger tasks and merge them into the existing suite
waza suggest skills/code-explainer --focus negative-triggers --count 5 --apply

# Write generated files to disk
waza suggest skills/code-explainer --apply

# Overwrite a previously generated suite
waza suggest skills/code-explainer --apply --force

# Print JSON-formatted suggestion payload
waza suggest skills/code-explainer --format json
```

### `waza tokens count [paths...]`

Count tokens in markdown files. Paths may be files or directories (scanned recursively for `.md`/`.mdx`).

| Flag | Description |
|------|-------------|
| `--format <fmt>` | Output format: `table` or `json` (default: `table`) |
| `--sort <field>` | Sort by: `tokens`, `name`, or `path` (default: `path`) |
| `--min-tokens <n>` | Filter files below n tokens |
| `--no-total` | Hide total row in table output |

### `waza tokens compare [refs...]`

Compare markdown token counts between git refs.

With no arguments, compares HEAD to the working tree.
With one ref, compares that ref to the working tree.
With two refs, compares the first ref to the second.

| Flag | Description |
|------|-------------|
| `--format <fmt>` | Output format: `table` or `json` (default: `table`) |
| `--show-unchanged` | Include unchanged files in output |
| `--strict` | Exit with code 1 if any file exceeds its absolute token limit |
| `--skills` | Only compare SKILL.md files under configured skill roots |
| `--threshold <n>` | Fail when any existing file increases by more than n percent (0 = disabled) |

Use `--skills` to restrict comparison to SKILL.md files under configured skill
roots (`skills/`, `.github/skills/`, APM `.apm/skills/` outputs, and
`paths.skills` from `.waza.yaml`). In skills mode the default base ref is
`origin/main` (falling back to `main`).

Use `--threshold` for CI gating — newly added files are exempt from threshold
checks (no baseline) but still subject to absolute limit checks with `--strict`.

```bash
# Compare all markdown tokens between HEAD and working tree
waza tokens compare

# Skill-aware comparison vs main with CI threshold
waza tokens compare main --skills --threshold 10

# JSON output for CI pipelines
waza tokens compare main --skills --threshold 10 --strict --format json
```

### `waza tokens profile [skill-name | path]`

Structural analysis of SKILL.md files — reports token count, section count, code block count, and workflow step detection with a one-line summary and warnings.

| Flag | Description |
|------|-------------|
| `--format <fmt>` | Output format: `text` or `json` (default: `text`) |
| `--tokenizer <t>` | Tokenizer: `bpe` or `estimate` (default: `bpe`) |

**Example output:**
```
📊 my-skill: 1,722 tokens (detailed ✓), 8 sections, 4 code blocks
   ⚠️  no workflow steps detected
```

### `waza tokens suggest [paths...]`

Suggest ways to reduce token usage in markdown files. Paths may be files or
directories (scanned recursively for `.md`/`.mdx`).

| Flag | Description |
|------|-------------|
| `--format <fmt>` | Output format: `text` or `json` (default: `text`) |
| `--min-savings <n>` | Minimum estimated token savings for heuristic suggestions |
| `--copilot` | Enable Copilot-powered suggestions |
| `--model <id>` | Model to use with `--copilot` |

### `waza serve`

Start the waza dashboard server to visualize evaluation results. The HTTP server opens in your browser automatically and scans the specified directory for `.json` result files.

Optionally, run a JSON-RPC 2.0 server (for IDE integration) instead of the HTTP dashboard using the `--tcp` flag.

| Flag | Default | Description |
|------|---------|-------------|
| `--port <port>` | `3000` | HTTP server port |
| `--no-browser` | `false` | Don't auto-open the browser |
| `--results-dir <dir>` | `.` | Directory to scan for result files |
| `--tcp <addr>` | (off) | TCP address for JSON-RPC (e.g., `:9000`); defaults to loopback for security |
| `--tcp-allow-remote` | `false` | Allow TCP binding to non-loopback addresses (⚠️ no authentication) |

**Examples:**

Start the HTTP dashboard on port 3000:
```bash
waza serve
```

Start the HTTP dashboard on a custom port and scan a results directory:
```bash
waza serve --port 8080 --results-dir ./results
```

Start the dashboard without auto-opening the browser:
```bash
waza serve --no-browser
```

Start a JSON-RPC server for IDE integration:
```bash
waza serve --tcp :9000
```

**Dashboard Views:**

The dashboard displays evaluation results with:
- Task-level pass/fail status
- Raw resolved task prompts with JSON formatting and copy-to-clipboard
- Score distributions across trials
- Model comparisons
- Aggregated metrics and trends

For detailed documentation on the dashboard and result visualization, see [docs/GUIDE.md](https://github.com/microsoft/waza/blob/main/docs/GUIDE.md).

### `waza results`

Manage evaluation results stored in cloud or local storage.

#### `waza results list`

List all evaluation runs from configured cloud storage or local results directory.

| Flag | Description |
|------|-------------|
| `--limit <n>` | Maximum results to display (default: 20) |
| `--format <fmt>` | Output format: `table` or `json` (default: `table`) |

```bash
# List recent results
waza results list

# List with custom limit
waza results list --limit 20

# Output as JSON
waza results list --format json
```

#### `waza results compare <id1> <id2>`

Compare two evaluation runs side by side. Displays per-task score deltas, pass rate differences, and key metrics.

| Flag | Description |
|------|-------------|
| `--format <fmt>` | Output format: `table` or `json` (default: `table`) |

```bash
# Compare two runs
waza results compare run-20250226-001 run-20250226-002

# Output as JSON for further processing
waza results compare run-20250226-001 run-20250226-002 --format json
```

### `waza grade <eval.yaml>`

Run graders against agent output without executing an agent. Designed for standalone grading of previous eval runs.

| Flag | Description |
|------|-------------|
| `--task <id>` | Task ID to grade |
| `--results <file>` | Path to waza run output JSON |
| `--workspace <dir>` | Agent workspace directory for file-based graders; must point to the agent's actual workspace (default: `.`) |
| `--judge-model <model>` | Model for prompt graders |
| `-o, --output <file>` | Write full EvaluationOutcome JSON (compatible with `waza compare`) |
| `-v, --verbose` | Verbose output |

```bash
waza run eval.yaml --output results.json
waza grade eval.yaml --results results.json
```

### `waza session list`

List session event logs in a directory.

| Flag | Description |
|------|-------------|
| `--dir <dir>` | Directory to search for session logs (default: `.`) |

```bash
waza session list
waza session list --dir ./sessions
```

### `waza session view <session-file>`

Render a session timeline from an NDJSON event log.

```bash
waza session view session-2025-06-15.ndjson
```

## Cloud Storage

Waza can automatically upload evaluation results to Azure Blob Storage for team collaboration and historical tracking.

### Configuration

Add a `storage:` section to your `.waza.yaml`:

```yaml
storage:
  provider: azure-blob
  accountName: "myteamwaza"
  containerName: "waza-results"
  enabled: true
```

| Field | Description | Required |
|-------|-------------|----------|
| `provider` | Cloud provider (`azure-blob` currently supported) | Yes |
| `accountName` | Azure Storage account name | Yes |
| `containerName` | Blob container name (default: `waza-results`) | No |
| `enabled` | Enable/disable uploads (default: `true` when configured) | No |

### Authentication

Waza uses **DefaultAzureCredential** — it automatically detects and uses available credentials in this order:

1. **Environment variables** (`AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`)
2. **Managed Identity** (on Azure services)
3. **Azure CLI** (`az login`)
4. **Visual Studio Code** (if signed in)
5. **Azure PowerShell** (if signed in)

In most cases, running `az login` is all you need:

```bash
az login
waza run eval.yaml  # Results auto-upload to Azure Storage
```

### How It Works

1. **Auto-upload on run:** When `storage:` is configured, `waza run` automatically uploads results to Azure Blob Storage
2. **Organized by skill:** Results are stored as `{skill-name}/{run-id}.json`
3. **Local copy kept:** Results are also saved locally (via `-o` flag)
4. **List remote results:** Use `waza results list` to browse uploaded runs
5. **Compare runs:** Use `waza results compare` to diff two remote results

### Example Workflow

```bash
# Configure once (edit .waza.yaml)
cat > .waza.yaml <<EOF
storage:
  provider: azure-blob
  accountName: "myteamwaza"
  containerName: "waza-results"
  enabled: true
EOF

# Authenticate
az login

# Run evaluations — results auto-upload
waza run evals/my-skill/eval.yaml -v

# Browse uploaded results
waza results list

# Compare two runs
waza results compare run-id-1 run-id-2
```

For step-by-step setup and troubleshooting, see [Getting Started with Azure Storage](https://github.com/microsoft/waza/blob/main/../docs/guides/azure-storage/) guide.

## Building

```bash
make build          # Compile binary to ./waza
make test           # Run tests with coverage
make lint           # Run golangci-lint
make fmt            # Format code and tidy modules
make install        # Install to GOPATH
```

## Project Structure

```
cmd/waza/              CLI entrypoint and command definitions
  tokens/              Token counting subcommand
internal/
  config/              Configuration with functional options
  execution/           AgentEngine interface (mock, copilot)
  graders/             Validator registry and built-in graders
  metrics/             Scoring metrics
  models/              Data structures (EvalSpec, TestCase, EvaluationOutcome)
  orchestration/       EvalRunner for coordinating execution
  reporting/           Result formatting and output
  transcript/          Per-task transcript capture
  wizard/              Interactive init wizard
examples/              Example eval suites
skills/                Example skills
```

## Eval Spec Format

```yaml
name: my-eval
skill: my-skill
schemaVersion: "1.2"
version: "1.0"

config:
  trials_per_task: 3
  max_attempts: 3          # Retry failed graders up to 3 times (default: 1, no retries)
  timeout_seconds: 300
  parallel: false
  executor: mock          # or copilot-sdk
  model: claude-sonnet-4-20250514
  group_by: model          # Group results by model (or other dimension)
  instruction_files:
    - .github/instructions/project.instructions.md

# Custom input variables available as {{.Vars.key}} in tasks and hooks
inputs:
  api_version: v2
  environment: production
  max_retries: 3

hooks:
  before_run:
    - command: "echo 'Starting evaluation'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

  after_run:
    - command: "echo 'Evaluation complete'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

  before_task:
    - command: "echo 'Running task: {{.TaskName}}'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

  after_task:
    - command: "echo 'Task {{.TaskName}} completed'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

mcp_mocks:
  - name: github
    tools:
      list_issues:
        input_schema:
          type: object
          required: [owner, repo]
        responses:
          - match:
              owner: microsoft
              repo: waza
            return:
              issues:
                - number: 363
                  title: MCP server mocks for hermetic eval
          - match_regex:
              repo: "^waza-.*"
            return:
              issues: []

graders:
  - ref: github.com/waza-evals/fact#factuality@v1.0.0
    name: factuality_strict
    weight: 2.0
    config:
      threshold: 0.9

  - type: text
    name: pattern_check
    config:
      regex_match: ["\\d+ tests passed"]

  - type: behavior
    name: efficiency
    config:
      max_tool_calls: 20
      max_duration_ms: 300000

  - type: action_sequence
    name: workflow_check
    config:
      matching_mode: in_order_match
      expected_actions: ["bash", "edit", "report_progress"]

# Task definitions: glob patterns or CSV dataset
tasks:
  - "tasks/*.yaml"

# Optional: Generate tasks from CSV dataset
# tasks_from: ./test-cases.csv
# range: [1, 10]  # Only include rows 1-10 (0-indexed, skips header)
```

`schemaVersion` uses `MAJOR.MINOR` format. Missing values are interpreted as the current schema version (currently `1.2`). Readers allow same-major minor additions with warnings for unknown fields, but reject different majors with a hint to run `waza migrate <file>`.

Remote grader refs use Go-module-style paths: `<host>/<owner>/<repo>[/path][#export]@<version>`. The remote module must provide a `waza.registry.yaml` manifest and export a grader preset. Config-only grader presets expand to built-in grader types by default; remote program graders require explicit trust with `waza registry add --allow-exec` or interactive confirmation. Run `waza get eval.yaml` after manually adding or changing refs so `waza.lock` records the resolved commit and digest.

`results.json` is currently emitted at `schemaVersion` `1.2`. Version `1.1` added per-turn checkpoints (`runs[].checkpoints[]`, see #358) and the normalized `runs[].tool_events[]` array (`turn`, `sequence`, `tool_call_id`, `tool_name`, `args`, `result`, `success`, `error`, `duration_ms`; see #366). Version `1.2` adds `runs[].snapshot_path` for `waza run --snapshot` artifacts (#367) and the eval-level `adversarial:` block consumed by `waza adversarial --spec` (#365). See [docs/PRD](https://github.com/microsoft/waza/blob/main/docs/PRD.md) and [schema-changes](https://github.com/microsoft/waza/blob/main/site/src/content/docs/reference/schema-changes.md) for details.

### MCP Mock Servers

Use top-level `mcp_mocks` with `schemaVersion: "1.1"` for deterministic Copilot SDK evals that need MCP tools without live services. Waza launches each mock as a local stdio MCP server, so CI runs do not need network ports, external credentials, or real service state. Waza exposes every tool declared by each mock to the Copilot CLI automatically; do not add a separate `tools` allowlist.

```yaml
schemaVersion: "1.1"
mcp_mocks:
  - name: github
    fixtures: fixtures/mcp/github
```

Inline responses support exact full-argument matching (`match`), JSON Schema matching (`match_schema`), and per-field regex matching (`match_regex`). Unknown tools and unmatched calls fail loudly with an MCP tool error that names the missing fixture.

### Adversarial Packs

Use top-level `adversarial` with `schemaVersion: "1.2"` to pin built-in fault-injection packs for `waza adversarial --spec`:

```yaml
schemaVersion: "1.2"
adversarial:
  packs:
    - prompt-injection
    - scope-bypass
  on_unsafe_outcome: fail
```

`on_unsafe_outcome: warn` records unsafe outcomes without failing the command.

### Custom Input Variables

Use the `inputs` section to define key-value variables available throughout your evaluation as `{{.Vars.key}}`:

```yaml
inputs:
  api_endpoint: https://api.example.com
  timeout: 30
  environment: staging

hooks:
  before_run:
    - command: "echo 'Testing against {{.Vars.environment}}'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false
```

Variables are accessible in:
- Hook commands
- Task prompts and fixtures (via template rendering)
- Grader configurations

### Instruction Files

Use `instruction_files` to apply `*.instructions.md` guidance during task execution:

```yaml
config:
  instruction_files:
    - .github/instructions/project.instructions.md
```

Instruction files are resolved from the active fixtures/context directory, copied into each task's temp workspace, and appended to the agent system message as path-labeled instructions. Task YAML files can also set top-level `instruction_files`; task-level entries are added to the eval-level list.

### Task Fixture Workspace

Task YAML can set `inputs.context.fixture` to copy a fixture file or directory into the fresh task workspace before the agent runs:

```yaml
inputs:
  context:
    fixture: fixtures/demo
  prompt: "Inspect repository files and summarize what you find."
```

Relative fixture paths are resolved from the eval spec directory. Directory fixtures are copied by contents into the workspace root.

### Skill Body Injection

By default, an eval with `skill: <name>` injects the target `SKILL.md` or `.agent.md` body into the agent system prompt. For trigger-precision evals, disable that body injection while preserving the skill association and SDK skill discovery:

```yaml
skill: xyz
config:
  inject_skill_body: false
```

The skill remains available to the Copilot SDK through its configured skill directories, but Waza does not add either the target `<skill_context>` block or a synthetic `<available_skills>` summary. `disabled_skills: ["*"]` still disables all skill loading.

### CSV Dataset Support

Generate tasks dynamically from a CSV file using `tasks_from`:

```yaml
# eval.yaml
tasks_from: ./test-cases.csv
range: [0, 50]  # Optional: limit to rows 0-50 (skip header at 0)
```

**CSV Format:**
```csv
prompt,expected_output,language
"Explain this function","Function explanation",python
"Review this code","Code review",javascript
```

**Task Generation:**
- **First row** is treated as column headers
- **Each subsequent row** becomes a task
- **Column values** are available as `{{.Vars.column_name}}`
- **Range filtering** (optional) allows limiting to a subset of rows

**Example task prompt using CSV variables:**

In your task file or inline prompt:
```yaml
prompt: "{{.Vars.prompt}}"
expected_output: "{{.Vars.expected_output}}"
language: "{{.Vars.language}}"
```

Tasks can also be mixed — use both explicit task files and CSV-generated tasks:

```yaml
tasks:
  - "tasks/*.yaml"        # Explicit tasks

tasks_from: ./test-cases.csv    # CSV-generated tasks
range: [0, 20]                  # Only first 20 rows
```

**CSV vs Inputs:**
- `inputs`: Static key-value pairs defined once in eval.yaml
- `tasks_from`: Generates multiple tasks from CSV rows
- **Conflict resolution**: CSV column values override `inputs` for the same key

### Retry/Attempts

Use `max_attempts` to retry failed grader validations within each trial:

```yaml
config:
  max_attempts: 3  # Retry failed graders up to 3 times (default: 1, no retries)
```

When a grader fails, waza will retry the task execution up to `max_attempts` times. The evaluation outcome includes an `attempts` field showing how many executions were needed to pass. This is useful for handling transient failures in external services or non-deterministic grader behavior.

**Output:** JSON results include `attempts` per task showing the number of executions performed.

### Grouping Results

Use `group_by` to organize results by a dimension (e.g., model, environment). Results are grouped in CLI output and JSON results include group statistics:

```yaml
config:
  group_by: model
```

Grouped results in JSON output include `GroupStats`:
```json
{
  "group_stats": [
    {
      "name": "claude-sonnet-4-20250514",
      "passed": 8,
      "total": 10,
      "avg_score": 0.85
    }
  ]
}
```

### Lifecycle Hooks

Use `hooks` to run commands before/after evaluations and tasks:

```yaml
hooks:
  before_run:
    - command: "npm install"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: true

  after_run:
    - command: "rm -rf node_modules"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

  before_task:
    - command: "echo 'Task: {{.TaskName}}'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false

  after_task:
    - command: "echo 'Done: {{.TaskName}}'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false
```

**Hook Fields:**
- `command` — Shell command to execute
- `working_directory` — Directory to run command in (relative to eval.yaml)
- `exit_codes` — List of acceptable exit codes (default: `[0]`)
- `error_on_fail` — Fail entire evaluation if hook fails (default: `false`)

**Lifecycle Points:**
- `before_run` — Execute once before all tasks
- `after_run` — Execute once after all tasks
- `before_task` — Execute before each task
- `after_task` — Execute after each task

**Template Variables in Hooks and Commands:**

Available variables in hook commands and task execution contexts:
- `{{.JobID}}` — Unique evaluation run identifier
- `{{.TaskName}}` — Name/ID of the current task (available in `before_task`/`after_task` only)
- `{{.Iteration}}` — Current trial number (1-indexed)
- `{{.Attempt}}` — Current attempt number (1-indexed, used for retries)
- `{{.Timestamp}}` — ISO 8601 timestamp of execution
- `{{.Vars.key}}` — User-defined variables from the `inputs` section or CSV columns

Custom variables can be defined in the `inputs` section and referenced in hooks:

```yaml
inputs:
  environment: production
  api_version: v2
  debug_mode: "true"

hooks:
  before_run:
    - command: "echo 'Starting eval {{.JobID}} in {{.Vars.environment}}'"
      working_directory: "."
      exit_codes: [0]
      error_on_fail: false
```

When using CSV-generated tasks, each row's column values are also available as `{{.Vars.column_name}}`.

## Git Repository Resources

Tasks can materialize a clean copy of a local git repository into the per-task workspace before the agent runs. This is most useful when you are developing skills inside the same repository the skills are intended to operate on (for example, the `eng/` tooling in `azure-sdk-for-rust`): rather than hand-staging fixtures, point at the local clone and each test run gets an isolated checkout.

Today only the `worktree` strategy is supported. It uses `git worktree add --detach` against a local clone — cheap (shares the same `.git` object store), no network required, and branch/tag names won't conflict with the source repo's current checkout.

```yaml
# task.yaml
id: my-task
name: Repo-aware task
inputs:
  prompt: "Explain the layout of this repository"
  workdir: my-repo          # optional: where the agent starts (relative to workspace)
  repos:
    - type: worktree        # required; only "worktree" is currently supported
      source: /path/to/local/clone   # required; local git repo to source from
      commit: main          # optional; commit SHA, branch, or tag (defaults to HEAD)
      dest: my-repo         # optional; subdir under workspace (omit to use workspace root)
```

**Fields:**

| Field    | Required | Description |
|---|---|---|
| `type`   | yes | Materialization strategy. Currently only `worktree`. |
| `source` | yes | Local filesystem path to a git repository to source the checkout from. |
| `commit` | no  | Commit SHA, branch, or tag. Defaults to the source repo's HEAD. Branch/tag names use `--detach` so they don't conflict with the source checkout. |
| `dest`   | yes | Relative subdirectory under the workspace where the repo is materialized. Required because `git worktree add` refuses targets that already exist, and the workspace root is created up-front. Must not contain `..` segments. |

`workdir` (also under `inputs`) is an optional relative path inside the workspace to use as the agent's working directory — typically set to the same value as `dest` so the agent starts inside the checked-out repo.

Waza automatically removes each worktree on engine shutdown (`git worktree remove --force`) before deleting the workspace directory, so the source repo's `.git/worktrees/` bookkeeping stays clean.

**Out of scope today** (tracked separately): HTTPS / SSH clone strategies, submodules, Git LFS, and auto-detecting "the repo this test is running in" without an explicit `source`.

## Responder (interactive skills)

For skills that ask follow-up questions, configure a `responder` — an LLM that plays the user and answers the skill's questions. It is mutually exclusive with `follow_up_prompts`.

```yaml
# task.yaml
inputs:
  prompt: "Add a new agent to my application"
  responder:
    model: gpt-4o          # optional; defaults to config.model
    instructions: |
      The agent you want is "research-agent" with system instructions
      "Search the web and summarise findings", tools web_search + url_fetch,
      and no handoffs. Answer the skill's questions consistently with this.
      If you genuinely can't infer an answer, abstain.
    max_followups: 8
```

After each agent turn the responder either **replies** (the answer is sent back, continuing the conversation), **stops** (the agent is done), or **abstains** — which fails the run with a distinct `abstained` outcome, signalling the brief is too vague. If `max_followups` is reached while the agent is still asking questions, the loop stops with outcome `cap_exhausted` and graders evaluate the final state. Each task carries its own responder, so the same skill can be tested against several target configurations.

**Fields** (under `inputs.responder`):

| Field           | Required | Description |
|---|---|---|
| `instructions`  | yes | The target configuration the responder represents and the rule for abstaining. |
| `max_followups` | yes | Maximum number of responder replies before the loop stops (`>= 1`). |
| `model`         | no  | Model used for the responder LLM. Defaults to the eval-level `config.model`. |

## CI/CD Integration

Waza is designed to work seamlessly with CI/CD pipelines.

### Integrating Waza in CI

Waza can validate your skill in CI before publishing:

#### Installation in CI

**Option 1: Binary install (recommended)**
```bash
curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash
```

**Option 2: Install from source**
```bash
# Requires Go 1.26+ and Git LFS
git clone https://github.com/microsoft/waza.git
cd waza
git lfs install
git lfs pull
go build -o waza ./cmd/waza
```

**Option 3: Use Docker**
```bash
docker build -t waza:local .
docker run -v $(pwd):/workspace waza:local run eval/eval.yaml
```

#### Quick Workflow Setup

Copy [`.github/workflows/skills-ci-example.yml`](https://github.com/microsoft/waza/blob/main/.github/workflows/skills-ci-example.yml) to your skill repository:

```yaml
jobs:
  evaluate-skill:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install waza
        run: curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash
      - run: waza run eval/eval.yaml --verbose --output results.json
      - uses: actions/upload-artifact@v4
        with:
          name: waza-evaluation-results
          path: results.json
```

#### Environment Requirements

| Requirement | Details |
|-------------|---------|
| **Go Version** | 1.26 or higher |
| **Executor** | Use `mock` executor for CI (no API keys needed) |
| **Copilot Auth** | Required for the default `copilot-sdk` route; set `GITHUB_TOKEN` in CI. Custom providers can be configured with `COPILOT_BASE_URL` or `COPILOT_PROVIDER_BASE_URL` instead. |
| **Exit Codes** | 0=success, 1=test failure, 2=config error |

#### Expected Skill Structure

```
your-skill/
├── SKILL.md              # Skill definition
└── eval/                 # Evaluation suite
    ├── eval.yaml         # Benchmark spec
    ├── tasks/            # Task definitions
    │   └── *.yaml
    └── fixtures/         # Context files
        ├── .github/
        │   └── instructions/
        │       └── project.instructions.md
        └── *.txt
```

#### Custom Copilot SDK Providers

By default, the `copilot-sdk` executor uses GitHub Copilot and requires Copilot authentication. To route sessions through a custom provider supported by the Copilot SDK, set a provider base URL before running Waza:

```bash
COPILOT_BASE_URL=https://waza-test-resource.openai.azure.com \
COPILOT_PROVIDER=azure \
COPILOT_WIRE_API=responses \
waza run eval/eval.yaml --executor copilot-sdk --model my-model
```

Supported environment variables:

| Variable | Description |
|----------|-------------|
| `COPILOT_BASE_URL` or `COPILOT_PROVIDER_BASE_URL` | Custom provider endpoint. When set, Waza skips the Copilot auth check and passes provider config to the SDK. |
| `COPILOT_PROVIDER` or `COPILOT_PROVIDER_TYPE` | Provider type passed through to the SDK. |
| `COPILOT_WIRE_API` or `COPILOT_PROVIDER_WIRE_API` | Wire format passed through to the SDK, for example `responses` or `completions`, depending on provider. |
| `COPILOT_API_KEY` or `COPILOT_PROVIDER_API_KEY` | API key for the custom provider, if required. |
| `COPILOT_BEARER_TOKEN` or `COPILOT_PROVIDER_BEARER_TOKEN` | Bearer token for the custom provider, if required. |

When a custom provider is active, the CLI usage summary labels the SDK request counter as `Provider Requests` instead of `Premium Requests`. Result JSON records `usage.provider: "custom"` and a sanitized `usage.provider_host`; it does not store the full provider URL.

### For Waza Repository

This repository includes reusable workflows:

1. **[`.github/workflows/waza-eval.yml`](https://github.com/microsoft/waza/blob/main/.github/workflows/waza-eval.yml)** - Reusable workflow for running evals
   ```yaml
   jobs:
     eval:
       uses: ./.github/workflows/waza-eval.yml
       with:
         eval-yaml: 'examples/code-explainer/eval.yaml'
         verbose: true
   ```

2. **[`examples/ci/eval-on-pr.yml`](https://github.com/microsoft/waza/blob/main/examples/ci/eval-on-pr.yml)** - Matrix testing across models

3. **[`examples/ci/basic-example.yml`](https://github.com/microsoft/waza/blob/main/examples/ci/basic-example.yml)** - Minimal workflow example

4. **[`.github/workflows/weekly-regression-loop.yml`](https://github.com/microsoft/waza/blob/main/.github/workflows/weekly-regression-loop.yml)** - Scheduled regression detection that archives dated artifacts and upserts follow-up issues on regressions

5. **[`.github/workflows/auto-merge.yml`](https://github.com/microsoft/waza/blob/main/.github/workflows/auto-merge.yml)** - Safe auto-merge gate for trusted, labeled PRs (`agent-merge` label on PRs targeting `main`)

See [`examples/ci/README.md`](https://github.com/microsoft/waza/blob/main/examples/ci/README.md) for detailed documentation and more examples.

### Available Grader Types

Waza supports multiple grader types for comprehensive evaluation:

| Grader | Purpose | Documentation |
|--------|---------|---------------|
| `code` | Python/JavaScript assertion-based validation | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#code---assertion-based-grader) |
| `text` | Substring and pattern matching in output | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#text---text-matching-grader) |
| `file` | File existence and content validation | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#file---file-system-validation) |
| `diff` | Workspace file comparison with snapshots and fragments | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#diff---workspace-file-comparison) |
| `behavior` | Agent behavior constraints (tool calls, tokens, duration) | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#behavior---agent-behavior-validation) |
| `action_sequence` | Tool call sequence validation with F1 scoring | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#action_sequence---tool-call-sequence-validation) |
| `skill_invocation` | Skill orchestration sequence validation | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#skill_invocation---skill-invocation-sequence-validation) |
| `prompt` | LLM-as-judge evaluation with rubrics (built-in: `groundedness`, `helpfulness`, `instruction-following`, `refusal-correctness`, `tool-use-appropriateness`) | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#prompt---llm-based-evaluation) |
| `trigger_tests` | Prompt trigger accuracy detection | [docs/GRADERS.md](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md#trigger-tests) |

See the complete [Grader Reference](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md) for detailed configuration options and examples.

## Documentation

- **[Getting Started](https://github.com/microsoft/waza/blob/main/docs/GETTING-STARTED.md)** - Complete walkthrough: init → new → run → check
- **[Demo Guide](https://github.com/microsoft/waza/blob/main/docs/DEMO-GUIDE.md)** - 7 live demo scenarios for presentations
- **[Grader Reference](https://github.com/microsoft/waza/blob/main/docs/GRADERS.md)** - Complete grader types and configuration
- **[Tutorial](https://github.com/microsoft/waza/blob/main/docs/TUTORIAL.md)** - Getting started with writing skill evals
- **[CI Integration](https://github.com/microsoft/waza/blob/main/docs/SKILLS_CI_INTEGRATION.md)** - GitHub Actions workflows for skill evaluation
- **[Token Management](https://github.com/microsoft/waza/blob/main/docs/TOKEN-LIMITS.md)** - Tracking and optimizing skill context size

## Contributing

See [AGENTS.md](https://github.com/microsoft/waza/blob/main/AGENTS.md) for coding guidelines.

- Use [conventional commits](https://www.conventionalcommits.org/) (`feat:`, `fix:`, `docs:`, etc.)
- Go CI is required: `Build and Test Go Implementation` and `Lint Go Code` must pass
- Add tests for new features
- Update docs when changing CLI surface

## Legacy Python Implementation

The Python implementation has been superseded by the Go CLI. The last Python release is available at [v0.3.2](https://github.com/microsoft/waza/releases/tag/v0.3.2). Starting with v0.4.0-alpha.1, waza is distributed exclusively as pre-built Go binaries.

## License

See [LICENSE](https://github.com/microsoft/waza/blob/main/LICENSE).

---

## Part: Skills

---

<!-- chapter:begin slug=agent-collaboration position=1 -->

## 1. agent-collaboration

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/agent-collaboration/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/agent-collaboration/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/agent-collaboration.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "agent-collaboration"
description: "Standard collaboration patterns for all squad agents — worktree awareness, decisions, cross-agent communication"
domain: "team-workflow"
confidence: "high"
source: "extracted from charter boilerplate — identical content in 18+ agent charters"
---

## Context

Every agent on the team follows identical collaboration patterns for worktree awareness, decision recording, and cross-agent communication. These were previously duplicated in every charter's Collaboration section (~300 bytes × 18 agents = ~5.4KB of redundant context). Now centralized here.

The coordinator's spawn prompt already instructs agents to read decisions.md and their history.md. This skill adds the patterns for WRITING decisions and requesting help.

## Patterns

### Worktree Awareness
Use the `TEAM ROOT` path provided in your spawn prompt. All `.squad/` paths are relative to this root. If TEAM ROOT is not provided (rare), run `git rev-parse --show-toplevel` as fallback. Never assume CWD is the repo root.

### Decision Recording
After making a decision that affects other team members, write it to:
`.squad/decisions/inbox/{your-name}-{brief-slug}.md`

Format:
```
### {date}: {decision title}
**By:** {Your Name}
**What:** {the decision}
**Why:** {rationale}
```

### Cross-Agent Communication
If you need another team member's input, say so in your response. The coordinator will bring them in. Don't try to do work outside your domain.

### Reviewer Protocol
If you have reviewer authority and reject work: the original author is locked out from revising that artifact. A different agent must own the revision. State who should revise in your rejection response.

## Anti-Patterns
- Don't read all agent charters — you only need your own context + decisions.md
- Don't write directly to `.squad/decisions.md` — always use the inbox drop-box
- Don't modify other agents' history.md files — that's Scribe's job
- Don't assume CWD is the repo root — always use TEAM ROOT

<!-- chapter:end slug=agent-collaboration -->

---

<!-- chapter:begin slug=error-recovery position=2 -->

## 2. error-recovery

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/error-recovery/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/error-recovery/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/error-recovery.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "error-recovery"
description: "Standard recovery patterns for all squad agents. When something fails, adapt — don't just report the failure."
domain: "reliability, agent-coordination"
confidence: "high"
license: MIT
---

# Error Recovery Patterns

Standard recovery patterns for all squad agents. When something fails, **adapt** — don't just report the failure.

---

## 1. Retry with Backoff

**When:** Transient failures — API timeouts, rate limits, network errors, temporary service unavailability.

**Pattern:**
1. Wait briefly, then retry (start at 2s, double each attempt)
2. Maximum 3 retries before escalating
3. Log each attempt with the error received

**Example:** API call returns 429 Too Many Requests → wait 2s → retry → wait 4s → retry → wait 8s → retry → escalate if still failing.

---

## 2. Fallback Alternatives

**When:** Primary tool or approach fails and an alternative exists.

**Pattern:**
1. Attempt primary approach
2. On failure, identify alternative tool/method
3. Try the alternative with the same intent
4. Document which alternative was used and why

**Example:** Primary CLI tool fails → fall back to direct API call for the same operation.

---

## 3. Diagnose-and-Fix

**When:** Build failures, test failures, linting errors — structured errors with actionable output.

**Pattern:**
1. Read the full error output carefully
2. Identify the root cause from error messages
3. Attempt a targeted fix
4. Re-run to verify the fix
5. Maximum 3 fix-retry cycles before escalating

**Example:** Build fails with a type error → check for missing import → add it → rebuild.

---

## 4. Escalate with Context

**When:** Recovery attempts have been exhausted, or the failure requires human judgment.

**Pattern:**
1. Summarize what was attempted and what failed
2. Include the exact error messages
3. State what you believe the root cause is
4. Suggest next steps or who might be able to help
5. Hand off to the coordinator or the appropriate specialist

**Example:** After 3 failed build attempts → "Build fails on line 42 with null reference. Tried X, Y, Z. Likely a design issue in the Foo module. Recommend the code owner review."

---

## 5. Graceful Degradation

**When:** A non-critical step fails but the overall task can still deliver value.

**Pattern:**
1. Determine if the failed step is critical to the task outcome
2. If non-critical, log the failure and continue
3. Deliver partial results with a clear note of what was skipped
4. Offer to retry the skipped step separately

**Example:** Generating a report with 5 sections — section 3 data source is unavailable → produce the report with 4 sections, note that section 3 was skipped and why.

---

## Applying These Patterns

Each agent should reference these patterns in their charter's `## Error Recovery` section, tailored to their domain. The charter should list the agent's most common failure modes and map each to the appropriate pattern above.

**Selection guide:**

| Failure Type | Primary Pattern | Fallback Pattern |
|---|---|---|
| Network/API transient | Retry with Backoff | Escalate with Context |
| Tool/dependency missing | Fallback Alternatives | Escalate with Context |
| Build/test error | Diagnose-and-Fix | Escalate with Context |
| Auth/permissions | Retry with Backoff | Escalate with Context |
| Non-critical data missing | Graceful Degradation | — |
| Unknown/novel error | Escalate with Context | — |

<!-- chapter:end slug=error-recovery -->

---

<!-- chapter:begin slug=git-workflow position=3 -->

## 3. git-workflow

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/git-workflow/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/git-workflow/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/git-workflow.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "git-workflow"
description: "Squad branching model: dev-first workflow with insiders preview channel"
domain: "version-control"
confidence: "high"
source: "team-decision"
---

## Context

Squad uses a three-branch model. **All feature work starts from `dev`, not `main`.**

| Branch | Purpose | Publishes |
|--------|---------|-----------|
| `main` | Released, tagged, in-npm code only | `npm publish` on tag |
| `dev` | Integration branch — all feature work lands here | `npm publish --tag preview` on merge |
| `insiders` | Early-access channel — synced from dev | `npm publish --tag insiders` on sync |

## Branch Naming Convention

Issue branches MUST use: `squad/{issue-number}-{kebab-case-slug}`

Examples:
- `squad/195-fix-version-stamp-bug`
- `squad/42-add-profile-api`

## Workflow for Issue Work

1. **Branch from dev:**
   ```bash
   git checkout dev
   git pull origin dev
   git checkout -b squad/{issue-number}-{slug}
   ```

2. **Mark issue in-progress:**
   ```bash
   gh issue edit {number} --add-label "status:in-progress"
   ```

3. **Create draft PR targeting dev:**
   ```bash
   gh pr create --base dev --title "{description}" --body "Closes #{issue-number}" --draft
   ```

4. **Do the work.** Make changes, write tests, commit with issue reference.

5. **Push and mark ready:**
   ```bash
   git push -u origin squad/{issue-number}-{slug}
   gh pr ready
   ```

6. **After merge to dev:**
   ```bash
   git checkout dev
   git pull origin dev
   git branch -d squad/{issue-number}-{slug}
   git push origin --delete squad/{issue-number}-{slug}
   ```

## Parallel Multi-Issue Work (Worktrees)

When the coordinator routes multiple issues simultaneously (e.g., "fix bugs X, Y, and Z"), use `git worktree` to give each agent an isolated working directory. No filesystem collisions, no branch-switching overhead.

### When to Use Worktrees vs Sequential

| Scenario | Strategy |
|----------|----------|
| Single issue | Standard workflow above — no worktree needed |
| 2+ simultaneous issues in same repo | Worktrees — one per issue |
| Work spanning multiple repos | Separate clones as siblings (see Multi-Repo below) |

### Setup

From the main clone (must be on dev or any branch):

```bash
# Ensure dev is current
git fetch origin dev

# Create a worktree per issue — siblings to the main clone
git worktree add ../squad-195 -b squad/195-fix-stamp-bug origin/dev
git worktree add ../squad-193 -b squad/193-refactor-loader origin/dev
```

**Naming convention:** `../{repo-name}-{issue-number}` (e.g., `../squad-195`, `../squad-pr-42`).

Each worktree:
- Has its own working directory and index
- Is on its own `squad/{issue-number}-{slug}` branch from dev
- Shares the same `.git` object store (disk-efficient)

### Per-Worktree Agent Workflow

Each agent operates inside its worktree exactly like the single-issue workflow:

```bash
cd ../squad-195

# Work normally — commits, tests, pushes
git add -A && git commit -m "fix: stamp bug (#195)"
git push -u origin squad/195-fix-stamp-bug

# Create PR targeting dev
gh pr create --base dev --title "fix: stamp bug" --body "Closes #195" --draft
```

All PRs target `dev` independently. Agents never interfere with each other's filesystem.

### .squad/ State in Worktrees

The `.squad/` directory exists in each worktree as a copy. This is safe because:
- `.gitattributes` declares `merge=union` on append-only files (history.md, decisions.md, logs)
- Each agent appends to its own section; union merge reconciles on PR merge to dev
- **Rule:** Never rewrite or reorder `.squad/` files in a worktree — append only

### Cleanup After Merge

After a worktree's PR is merged to dev:

```bash
# From the main clone
git worktree remove ../squad-195
git worktree prune          # clean stale metadata
git branch -d squad/195-fix-stamp-bug
git push origin --delete squad/195-fix-stamp-bug
```

If a worktree was deleted manually (rm -rf), `git worktree prune` recovers the state.

---

## Multi-Repo Downstream Scenarios

When work spans multiple repositories (e.g., squad-cli changes need squad-sdk changes, or a user's app depends on squad):

### Setup

Clone downstream repos as siblings to the main repo:

```
~/work/
  squad-pr/          # main repo
  squad-sdk/         # downstream dependency
  user-app/          # consumer project
```

Each repo gets its own issue branch following its own naming convention. If the downstream repo also uses Squad conventions, use `squad/{issue-number}-{slug}`.

### Coordinated PRs

- Create PRs in each repo independently
- Link them in PR descriptions:
  ```
  Closes #42

  **Depends on:** squad-sdk PR #17 (squad-sdk changes required for this feature)
  ```
- Merge order: dependencies first (e.g., squad-sdk), then dependents (e.g., squad-cli)

### Local Linking for Testing

Before pushing, verify cross-repo changes work together:

```bash
# Node.js / npm
cd ../squad-sdk && npm link
cd ../squad-pr && npm link squad-sdk

# Go
# Use replace directive in go.mod:
# replace github.com/org/squad-sdk => ../squad-sdk

# Python
cd ../squad-sdk && pip install -e .
```

**Important:** Remove local links before committing. `npm link` and `go replace` are dev-only — CI must use published packages or PR-specific refs.

### Worktrees + Multi-Repo

These compose naturally. You can have:
- Multiple worktrees in the main repo (parallel issues)
- Separate clones for downstream repos
- Each combination operates independently

---

## Anti-Patterns

- ❌ Branching from main (branch from dev)
- ❌ PR targeting main directly (target dev)
- ❌ Non-conforming branch names (must be squad/{number}-{slug})
- ❌ Committing directly to main or dev (use PRs)
- ❌ Switching branches in the main clone while worktrees are active (use worktrees instead)
- ❌ Using worktrees for cross-repo work (use separate clones)
- ❌ Leaving stale worktrees after PR merge (clean up immediately)

## Promotion Pipeline

- dev → insiders: Automated sync on green build
- dev → main: Manual merge when ready for stable release, then tag
- Hotfixes: Branch from main as `hotfix/{slug}`, PR to dev, cherry-pick to main if urgent

<!-- chapter:end slug=git-workflow -->

---

<!-- chapter:begin slug=reviewer-protocol position=4 -->

## 4. reviewer-protocol

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/reviewer-protocol/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/reviewer-protocol/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/reviewer-protocol.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "reviewer-protocol"
description: "Reviewer rejection workflow and strict lockout semantics"
domain: "orchestration"
confidence: "high"
source: "extracted"
---

## Context

When a team member has a **Reviewer** role (e.g., Tester, Code Reviewer, Lead), they may approve or reject work from other agents. On rejection, the coordinator enforces strict lockout rules to ensure the original author does NOT self-revise. This prevents defensive feedback loops and ensures independent review.

## Patterns

### Reviewer Rejection Protocol

When a team member has a **Reviewer** role:

- Reviewers may **approve** or **reject** work from other agents.
- On **rejection**, the Reviewer may choose ONE of:
  1. **Reassign:** Require a *different* agent to do the revision (not the original author).
  2. **Escalate:** Require a *new* agent be spawned with specific expertise.
- The Coordinator MUST enforce this. If the Reviewer says "someone else should fix this," the original agent does NOT get to self-revise.
- If the Reviewer approves, work proceeds normally.

### Strict Lockout Semantics

When an artifact is **rejected** by a Reviewer:

1. **The original author is locked out.** They may NOT produce the next version of that artifact. No exceptions.
2. **A different agent MUST own the revision.** The Coordinator selects the revision author based on the Reviewer's recommendation (reassign or escalate).
3. **The Coordinator enforces this mechanically.** Before spawning a revision agent, the Coordinator MUST verify that the selected agent is NOT the original author. If the Reviewer names the original author as the fix agent, the Coordinator MUST refuse and ask the Reviewer to name a different agent.
4. **The locked-out author may NOT contribute to the revision** in any form — not as a co-author, advisor, or pair. The revision must be independently produced.
5. **Lockout scope:** The lockout applies to the specific artifact that was rejected. The original author may still work on other unrelated artifacts.
6. **Lockout duration:** The lockout persists for that revision cycle. If the revision is also rejected, the same rule applies again — the revision author is now also locked out, and a third agent must revise.
7. **Deadlock handling:** If all eligible agents have been locked out of an artifact, the Coordinator MUST escalate to the user rather than re-admitting a locked-out author.

## Examples

**Example 1: Reassign after rejection**
1. Fenster writes authentication module
2. Hockney (Tester) reviews → rejects: "Error handling is missing. Verbal should fix this."
3. Coordinator: Fenster is now locked out of this artifact
4. Coordinator spawns Verbal to revise the authentication module
5. Verbal produces v2
6. Hockney reviews v2 → approves
7. Lockout clears for next artifact

**Example 2: Escalate for expertise**
1. Edie writes TypeScript config
2. Keaton (Lead) reviews → rejects: "Need someone with deeper TS knowledge. Escalate."
3. Coordinator: Edie is now locked out
4. Coordinator spawns new agent (or existing TS expert) to revise
5. New agent produces v2
6. Keaton reviews v2

**Example 3: Deadlock handling**
1. Fenster writes module → rejected
2. Verbal revises → rejected
3. Hockney revises → rejected
4. All 3 eligible agents are now locked out
5. Coordinator: "All eligible agents have been locked out. Escalating to user: [artifact details]"

**Example 4: Reviewer accidentally names original author**
1. Fenster writes module → rejected
2. Hockney says: "Fenster should fix the error handling"
3. Coordinator: "Fenster is locked out as the original author. Please name a different agent."
4. Hockney: "Verbal, then"
5. Coordinator spawns Verbal

## Anti-Patterns

- ❌ Allowing the original author to self-revise after rejection
- ❌ Treating the locked-out author as an "advisor" or "co-author" on the revision
- ❌ Re-admitting a locked-out author when deadlock occurs (must escalate to user)
- ❌ Applying lockout across unrelated artifacts (scope is per-artifact)
- ❌ Accepting the Reviewer's assignment when they name the original author (must refuse and ask for a different agent)
- ❌ Clearing lockout before the revision is approved (lockout persists through revision cycle)
- ❌ Skipping verification that the revision agent is not the original author

<!-- chapter:end slug=reviewer-protocol -->

---

<!-- chapter:begin slug=secret-handling position=5 -->

## 5. secret-handling

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/secret-handling/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/secret-handling/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/secret-handling.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: secret-handling
description: Never read .env files or write secrets to .squad/ committed files
domain: security, file-operations, team-collaboration
confidence: high
source: earned (issue #267 — credential leak incident)
---

## Context

Spawned agents have read access to the entire repository, including `.env` files containing live credentials. If an agent reads secrets and writes them to `.squad/` files (decisions, logs, history), Scribe auto-commits them to git, exposing them in remote history. This skill codifies absolute prohibitions and safe alternatives.

## Patterns

### Prohibited File Reads

**NEVER read these files:**
- `.env` (production secrets)
- `.env.local` (local dev secrets)
- `.env.production` (production environment)
- `.env.development` (development environment)
- `.env.staging` (staging environment)
- `.env.test` (test environment with real credentials)
- Any file matching `.env.*` UNLESS explicitly allowed (see below)

**Allowed alternatives:**
- `.env.example` (safe — contains placeholder values, no real secrets)
- `.env.sample` (safe — documentation template)
- `.env.template` (safe — schema/structure reference)

**If you need config info:**
1. **Ask the user directly** — "What's the database connection string?"
2. **Read `.env.example`** — shows structure without exposing secrets
3. **Read documentation** — check `README.md`, `docs/`, config guides

**NEVER assume you can "just peek at .env to understand the schema."** Use `.env.example` or ask.

### Prohibited Output Patterns

**NEVER write these to `.squad/` files:**

| Pattern Type | Examples | Regex Pattern (for scanning) |
|--------------|----------|-------------------------------|
| API Keys | `OPENAI_API_KEY=sk-proj-...`, `GITHUB_TOKEN=ghp_...` | `[A-Z_]+(?:KEY|TOKEN|SECRET)=[^\s]+` |
| Passwords | `DB_PASSWORD=super_secret_123`, `password: "..."` | `(?:PASSWORD|PASS|PWD)[:=]\s*["']?[^\s"']+` |
| Connection Strings | `postgres://user:pass@host:5432/db`, `Server=...;Password=...` | `(?:postgres|mysql|mongodb)://[^@]+@|(?:Server|Host)=.*(?:Password|Pwd)=` |
| JWT Tokens | `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...` | `eyJ[A-Za-z0-9_-]+\.eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+` |
| Private Keys | `-----BEGIN PRIVATE KEY-----`, `-----BEGIN RSA PRIVATE KEY-----` | `-----BEGIN [A-Z ]+PRIVATE KEY-----` |
| AWS Credentials | `AKIA...`, `aws_secret_access_key=...` | `AKIA[0-9A-Z]{16}|aws_secret_access_key=[^\s]+` |
| Email Addresses | `user@example.com` (PII violation per team decision) | `[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}` |

**What to write instead:**
- Placeholder values: `DATABASE_URL=<set in .env>`
- Redacted references: `API key configured (see .env.example)`
- Architecture notes: "App uses JWT auth — token stored in session"
- Schema documentation: "Requires OPENAI_API_KEY, GITHUB_TOKEN (see .env.example for format)"

### Scribe Pre-Commit Validation

**Before committing `.squad/` changes, Scribe MUST:**

1. **Scan all staged files** for secret patterns (use regex table above)
2. **Check for prohibited file names** (don't commit `.env` even if manually staged)
3. **If secrets detected:**
   - STOP the commit (do NOT proceed)
   - Remove the file from staging: `git reset HEAD <file>`
   - Report to user:
     ```
     🚨 SECRET DETECTED — commit blocked
     
     File: .squad/decisions/inbox/river-db-config.md
     Pattern: DATABASE_URL=postgres://user:password@localhost:5432/prod
     
     This file contains credentials and MUST NOT be committed.
     Please remove the secret, replace with placeholder, and try again.
     ```
   - Exit with error (never silently skip)

4. **If no secrets detected:**
   - Proceed with commit as normal

**Implementation note for Scribe:**
- Run validation AFTER staging files, BEFORE calling `git commit`
- Use PowerShell `Select-String` or `git diff --cached` to scan staged content
- Fail loud — secret leaks are unacceptable, blocking the commit is correct behavior

### Remediation — If a Secret Was Already Committed

**If you discover a secret in git history:**

1. **STOP immediately** — do not make more commits
2. **Alert the user:**
   ```
   🚨 CREDENTIAL LEAK DETECTED
   
   A secret was found in git history:
   Commit: abc1234
   File: .squad/decisions/inbox/agent-config.md
   Pattern: API_KEY=sk-proj-...
   
   This requires immediate remediation:
   1. Revoke the exposed credential (regenerate API key, rotate password)
   2. Remove from git history (git filter-repo or BFG)
   3. Force-push the cleaned history
   
   Do NOT proceed with new work until this is resolved.
   ```
3. **Do NOT attempt to fix it yourself** — secret removal requires specialized tools
4. **Wait for user confirmation** before resuming work

## Examples

### ✓ Correct: Reading Config Schema

**Agent needs to know what environment variables are required:**

```
Agent: "What environment variables does this app need?"
→ Reads `.env.example`:
    OPENAI_API_KEY=sk-...
    DATABASE_URL=postgres://user:pass@localhost:5432/db
    REDIS_URL=redis://localhost:6379

→ Writes to .squad/decisions/inbox/river-env-setup.md:
    "App requires three environment variables:
    - OPENAI_API_KEY (OpenAI API key, format: sk-...)
    - DATABASE_URL (Postgres connection string)
    - REDIS_URL (Redis connection string)
    See .env.example for full schema."
```

### ✗ Incorrect: Reading Live Credentials

**Agent needs to know database schema:**

```
Agent: (reads .env)
    DATABASE_URL=postgres://admin:super_secret_pw@prod.example.com:5432/appdb

→ Writes to .squad/decisions/inbox/river-db-schema.md:
    "Database connection: postgres://admin:super_secret_pw@prod.example.com:5432/appdb"
    
🚨 VIOLATION: Live credential written to committed file
```

**Correct approach:**
```
Agent: (reads .env.example OR asks user)
User: "It's a Postgres database, schema is in migrations/"

→ Writes to .squad/decisions/inbox/river-db-schema.md:
    "Database: Postgres (connection configured in .env). Schema defined in db/migrations/."
```

### ✓ Correct: Scribe Pre-Commit Validation

**Scribe is about to commit:**

```powershell
# Stage files
git add .squad/

# Scan staged content for secrets
$stagedContent = git diff --cached
$secretPatterns = @(
    '[A-Z_]+(?:KEY|TOKEN|SECRET)=[^\s]+',
    '(?:PASSWORD|PASS|PWD)[:=]\s*["'']?[^\s"'']+',
    'eyJ[A-Za-z0-9_-]+\.eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+'
)

$detected = $false
foreach ($pattern in $secretPatterns) {
    if ($stagedContent -match $pattern) {
        $detected = $true
        Write-Host "🚨 SECRET DETECTED: $($matches[0])"
        break
    }
}

if ($detected) {
    # Remove from staging, report, exit
    git reset HEAD .squad/
    Write-Error "Commit blocked — secret detected in staged files"
    exit 1
}

# Safe to commit
git commit -F $msgFile
```

## Anti-Patterns

- ❌ Reading `.env` "just to check the schema" — use `.env.example` instead
- ❌ Writing "sanitized" connection strings that still contain credentials
- ❌ Assuming "it's just a dev environment" makes secrets safe to commit
- ❌ Committing first, scanning later — validation MUST happen before commit
- ❌ Silently skipping secret detection — fail loud, never silent
- ❌ Trusting agents to "know better" — enforce at multiple layers (prompt, hook, architecture)
- ❌ Writing secrets to "temporary" files in `.squad/` — Scribe commits ALL `.squad/` changes
- ❌ Extracting "just the host" from a connection string — still leaks infrastructure topology

<!-- chapter:end slug=secret-handling -->

---

<!-- chapter:begin slug=session-recovery position=6 -->

## 6. session-recovery

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/session-recovery/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/session-recovery/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/session-recovery.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "session-recovery"
description: "Find and resume interrupted Copilot CLI sessions using session_store queries"
domain: "workflow-recovery"
confidence: "high"
source: "earned"
tools:
  - name: "sql"
    description: "Query session_store database for past session history"
    when: "Always — session_store is the source of truth for session history"
---

## Context

Squad agents run in Copilot CLI sessions that can be interrupted — terminal crashes, network drops, machine restarts, or accidental window closes. When this happens, in-progress work may be left in a partially-completed state: branches with uncommitted changes, issues marked in-progress with no active agent, or checkpoints that were never finalized.

Copilot CLI stores session history in a SQLite database called `session_store` (read-only, accessed via the `sql` tool with `database: "session_store"`). This skill teaches agents how to query that store to detect interrupted sessions and resume work.

## Patterns

### 1. Find Recent Sessions

Query the `sessions` table filtered by time window. Include the last checkpoint to understand where the session stopped:

```sql
SELECT
  s.id,
  s.summary,
  s.cwd,
  s.branch,
  s.updated_at,
  (SELECT title FROM checkpoints
   WHERE session_id = s.id
   ORDER BY checkpoint_number DESC LIMIT 1) AS last_checkpoint
FROM sessions s
WHERE s.updated_at >= datetime('now', '-24 hours')
ORDER BY s.updated_at DESC;
```

### 2. Filter Out Automated Sessions

Automated agents (monitors, keep-alive, heartbeat) create high-volume sessions that obscure human-initiated work. Exclude them:

```sql
SELECT s.id, s.summary, s.cwd, s.updated_at,
  (SELECT title FROM checkpoints
   WHERE session_id = s.id
   ORDER BY checkpoint_number DESC LIMIT 1) AS last_checkpoint
FROM sessions s
WHERE s.updated_at >= datetime('now', '-24 hours')
  AND s.id NOT IN (
    SELECT DISTINCT t.session_id FROM turns t
    WHERE t.turn_index = 0
      AND (LOWER(t.user_message) LIKE '%keep-alive%'
           OR LOWER(t.user_message) LIKE '%heartbeat%')
  )
ORDER BY s.updated_at DESC;
```

### 3. Search by Topic (FTS5)

Use the `search_index` FTS5 table for keyword search. Expand queries with synonyms since this is keyword-based, not semantic:

```sql
SELECT DISTINCT s.id, s.summary, s.cwd, s.updated_at
FROM search_index si
JOIN sessions s ON si.session_id = s.id
WHERE search_index MATCH 'auth OR login OR token OR JWT'
  AND s.updated_at >= datetime('now', '-48 hours')
ORDER BY s.updated_at DESC
LIMIT 10;
```

### 4. Search by Working Directory

```sql
SELECT s.id, s.summary, s.updated_at,
  (SELECT title FROM checkpoints
   WHERE session_id = s.id
   ORDER BY checkpoint_number DESC LIMIT 1) AS last_checkpoint
FROM sessions s
WHERE s.cwd LIKE '%my-project%'
  AND s.updated_at >= datetime('now', '-48 hours')
ORDER BY s.updated_at DESC;
```

### 5. Get Full Session Context Before Resuming

Before resuming, inspect what the session was doing:

```sql
-- Conversation turns
SELECT turn_index, substr(user_message, 1, 200) AS ask, timestamp
FROM turns WHERE session_id = 'SESSION_ID' ORDER BY turn_index;

-- Checkpoint progress
SELECT checkpoint_number, title, overview
FROM checkpoints WHERE session_id = 'SESSION_ID' ORDER BY checkpoint_number;

-- Files touched
SELECT file_path, tool_name
FROM session_files WHERE session_id = 'SESSION_ID';

-- Linked PRs/issues/commits
SELECT ref_type, ref_value
FROM session_refs WHERE session_id = 'SESSION_ID';
```

### 6. Detect Orphaned Issue Work

Find sessions that were working on issues but may not have completed:

```sql
SELECT DISTINCT s.id, s.branch, s.summary, s.updated_at,
  sr.ref_type, sr.ref_value
FROM sessions s
JOIN session_refs sr ON s.id = sr.session_id
WHERE sr.ref_type = 'issue'
  AND s.updated_at >= datetime('now', '-48 hours')
ORDER BY s.updated_at DESC;
```

Cross-reference with `gh issue list --label "status:in-progress"` to find issues that are marked in-progress but have no active session.

### 7. Resume a Session

Once you have the session ID:

```bash
# Resume directly
copilot --resume SESSION_ID
```

## Examples

**Recovering from a crash during PR creation:**
1. Query recent sessions filtered by branch name
2. Find the session that was working on the PR
3. Check its last checkpoint — was the code committed? Was the PR created?
4. Resume or manually complete the remaining steps

**Finding yesterday's work on a feature:**
1. Use FTS5 search with feature keywords
2. Filter to the relevant working directory
3. Review checkpoint progress to see how far the session got
4. Resume if work remains, or start fresh with the context

## Anti-Patterns

- ❌ Searching by partial session IDs — always use full UUIDs
- ❌ Resuming sessions that completed successfully — they have no pending work
- ❌ Using `MATCH` with special characters without escaping — wrap paths in double quotes
- ❌ Skipping the automated-session filter — high-volume automated sessions will flood results
- ❌ Assuming FTS5 is semantic search — it's keyword-based; always expand queries with synonyms
- ❌ Ignoring checkpoint data — checkpoints show exactly where the session stopped

<!-- chapter:end slug=session-recovery -->

---

<!-- chapter:begin slug=squad-commands position=7 -->

## 7. squad-commands

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/squad-commands/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/squad-commands/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/squad-commands.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: squad-commands
description: >
  Categorized catalog of common Squad operations. Coordinator reads this
  file and presents it as an interactive menu when the user asks for
  available commands or help.
domain: squad-operations
confidence: high
source: first-party
triggers: ["squad commands", "what can squad do", "show me squad options", "slash commands"]
---

## Menu Presentation Rules

When the user triggers this skill ("squad commands", "help", "what can squad do", etc.):

1. **Category-level menu first.** Present category names as an `ask_user` choice list:
   ```
   📋 Squad Commands — pick a category:
   1. Install & Upgrade
   2. Team Management
   3. Issues & PRs
   4. Plugins & Skills
   5. Model & Cost
   6. Sessions & State
   ```
2. **Drill-down.** After selection, show operation titles in that category as a second `ask_user` list.
3. **Direct match skips the menu.** If the user says "how do I upgrade with state backend," match to the specific entry and go straight to argument collection.
4. **Compact fallback.** If `ask_user` is unavailable, render as a markdown table instead.
5. **Back / Cancel.** Include "← Back to categories" in sub-menus. Include "Cancel" in confirmation prompts. Respect "never mind" / "cancel" at any point.

**Argument collection:** For entries with `args`, iterate the list sequentially. Use `ask_user` with choices when `choices` is provided; free-text prompt otherwise. If the user says "just do it" or "defaults are fine," skip remaining args and use their defaults.

**Confirmation template:**
```
⚠️ This will {action-description}.
{what will change}
Proceed? (yes / no)
```

---

## Install & Upgrade

### Upgrade Squad CLI

- **intent:** upgrade squad, update squad, install latest version, get new version
- **summary:** Upgrade Squad CLI to the latest version for your channel
- **action:** shell
- **command:** squad upgrade
- **args:**
  - `state-backend`: Which state backend? | choices: {worktree, git-notes, orphan, two-layer} | default: (keep current)
- **confirm:** false
- **platform_caveats:** Requires terminal. In VS Code, open the integrated terminal and run the command directly.

### Initialize Squad

- **intent:** set up squad, initialize squad, create team, start squad in this project
- **summary:** Scaffold Squad in the current directory (idempotent)
- **action:** shell
- **command:** squad init
- **args:** (none)
- **confirm:** false
- **platform_caveats:** Requires terminal. Recommend a standalone terminal for best results.

### Switch State Backend

- **intent:** switch state backend, change state storage, use git-notes, use orphan branch
- **summary:** Change where Squad stores mutable state (config.json)
- **action:** file-edit
- **command:** .squad/config.json → stateBackend
- **args:**
  - `stateBackend`: Which state backend? | choices: {worktree, git-notes, orphan, two-layer} | default: (keep current)
- **confirm:** true
- **platform_caveats:** May require migration if switching away from worktree. Show current value and new value before confirming.

---

## Team Management

### Add Team Member

- **intent:** add team member, hire agent, add agent, add developer, recruit
- **summary:** Add a new agent to the team roster
- **action:** coordinator
- **command:** Add Team Member flow (Init Mode / Team Mode)
- **args:**
  - `role`: What role should this agent fill? (e.g., Frontend Dev, Backend Dev, QA Engineer)
  - `name`: Preferred name or casting universe? | default: (auto-cast from active universe)
- **confirm:** false

### Remove Team Member

- **intent:** remove team member, fire agent, delete agent, remove developer
- **summary:** Remove an agent and delete their charter and history files
- **action:** coordinator
- **command:** Remove Team Member flow
- **args:**
  - `member`: Which team member to remove? (name or role)
- **confirm:** true

### Reassign Roles

- **intent:** reassign role, change role, swap roles, update team member role
- **summary:** Update a team member's role in team.md and their charter
- **action:** coordinator
- **command:** Update team.md roster + charter.md
- **args:**
  - `member`: Which team member?
  - `newRole`: New role?
- **confirm:** false

### Show Roster

- **intent:** show roster, who is on the team, list team members, show team, capability profile
- **summary:** Display the current team roster and capability profile
- **action:** coordinator
- **command:** Direct Mode — read team.md, answer
- **args:** (none)
- **confirm:** false

---

## Issues & PRs

### Connect GitHub Repo

- **intent:** connect github, enable issues, set up issues, link repository, github issues mode
- **summary:** Connect this project to GitHub Issues via gh auth
- **action:** coordinator
- **command:** GitHub Issues Mode (connection flow)
- **args:** (none)
- **confirm:** false
- **platform_caveats:** Requires `gh auth login` to have been run in the terminal.

### Triage Issues

- **intent:** triage issues, review issues, assign issues, label issues
- **summary:** Run the Lead triage flow on open GitHub issues
- **action:** coordinator
- **command:** GitHub Issues Mode → Lead triage
- **args:** (none)
- **confirm:** false

### Activate Ralph

- **intent:** activate ralph, start ralph, ralph go, start work monitor, start auto-work
- **summary:** Activate Ralph — Work Monitor — to pick up and run queued issues
- **action:** coordinator
- **command:** Ralph — Work Monitor triggers
- **args:** (none)
- **confirm:** false

### Set Ralph Polling Interval

- **intent:** set ralph interval, change ralph timing, how often does ralph check, ralph every N minutes
- **summary:** Tell Ralph how frequently to poll for new work
- **action:** coordinator
- **command:** Ralph trigger: "Ralph, check every N minutes"
- **args:**
  - `interval`: How often should Ralph poll? (in minutes) | default: 10
- **confirm:** false

### Start Squad Watch

- **intent:** start watch, squad watch, monitor issues, watch for issues, auto-triage
- **summary:** Start squad watch to continuously poll and triage issues
- **action:** shell
- **command:** squad watch
- **args:**
  - `interval`: Poll interval in minutes | default: 10
- **confirm:** false
- **platform_caveats:** CLI-only — long-running foreground process. Not viable in VS Code without an integrated terminal. Run: `squad watch --interval {n}` in your terminal.

---

## Plugins & Skills

### Browse Plugin Marketplace

- **intent:** browse plugins, explore plugins, what plugins are available, plugin marketplace
- **summary:** Browse available plugins in the Squad marketplace
- **action:** shell
- **command:** squad plugin marketplace browse
- **args:**
  - `name`: Plugin name to search for | default: (browse all)
- **confirm:** false

### Add Marketplace Plugin

- **intent:** add plugin, install plugin, get plugin from marketplace
- **summary:** Add a plugin from the marketplace to this Squad
- **action:** shell
- **command:** squad plugin marketplace add
- **args:**
  - `plugin`: Plugin owner/repo (e.g., owner/plugin-name)
- **confirm:** false

### Remove Marketplace Plugin

- **intent:** remove plugin, uninstall plugin, delete plugin
- **summary:** Remove an installed marketplace plugin
- **action:** shell
- **command:** squad plugin marketplace remove
- **args:**
  - `name`: Plugin name to remove
- **confirm:** true

### List Marketplace Plugins

- **intent:** list plugins, show installed plugins, what plugins do I have
- **summary:** List all plugins registered in this Squad
- **action:** shell
- **command:** squad plugin marketplace list
- **args:** (none)
- **confirm:** false

### List Installed Skills

- **intent:** list skills, show skills, what skills are installed, skill catalog
- **summary:** List all skills installed in .squad/skills/ and .copilot/skills/
- **action:** coordinator
- **command:** Direct Mode — list .squad/skills/ and .copilot/skills/ directories
- **args:** (none)
- **confirm:** false

---

## Model & Cost

### Set Default Model

- **intent:** set default model, change model, use gpt-4, use claude, switch model
- **summary:** Set the default model for all agents in config.json
- **action:** file-edit
- **command:** .squad/config.json → defaultModel
- **args:**
  - `model`: Model name (e.g., gpt-4o, claude-sonnet-4.5, o3)
- **confirm:** false

### Override Per-Agent Model

- **intent:** set model for agent, agent model override, use different model for one agent
- **summary:** Set a model override for a specific agent in config.json
- **action:** file-edit
- **command:** .squad/config.json → agentModelOverrides.{agentName}
- **args:**
  - `agent`: Agent name (must match name in team.md)
  - `model`: Model name (e.g., gpt-4o, claude-sonnet-4.5)
- **confirm:** false

### Clear Model Preference

- **intent:** clear model, reset model, remove model preference, use default model
- **summary:** Remove a model override from config.json (reverts to system default)
- **action:** file-edit
- **command:** .squad/config.json → remove defaultModel or agentModelOverrides.{agentName}
- **args:**
  - `scope`: Clear default or a specific agent? | choices: {default model, specific agent} | default: default model
  - `agent`: Agent name (only if scope = specific agent)
- **confirm:** false

---

## Sessions & State

### Catch-Up Summary

- **intent:** catch me up, what happened, status, what did the team do, session summary
- **summary:** Summarize recent agent activity and key decisions
- **action:** coordinator
- **command:** Session catch-up flow (lazy scan)
- **args:** (none)
- **confirm:** false

### Show Recent Decisions

- **intent:** show decisions, recent decisions, what decisions were made, decision log
- **summary:** Display recent entries from .squad/decisions.md
- **action:** coordinator
- **command:** Direct Mode — read decisions.md, answer
- **args:** (none)
- **confirm:** false

### Archive Old Decisions

- **intent:** archive decisions, clean up decisions, move old decisions, compact decisions
- **summary:** Move old decisions from decisions.md to decisions-archive.md
- **action:** coordinator
- **command:** Move entries older than threshold from .squad/decisions.md → .squad/decisions-archive.md
- **args:**
  - `olderThan`: Archive decisions older than how many days? | default: 30
- **confirm:** true

### Summarize Agent History

- **intent:** summarize history, what did agent do, agent history, compress history
- **summary:** Spawn an agent to summarize and compress a team member's history file
- **action:** coordinator
- **command:** Spawn agent with history.md summarization task
- **args:**
  - `member`: Which team member's history to summarize?
- **confirm:** false

<!-- chapter:end slug=squad-commands -->

---

<!-- chapter:begin slug=squad-conventions position=8 -->

## 8. squad-conventions

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/squad-conventions/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/squad-conventions/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/squad-conventions.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "squad-conventions"
description: "Core conventions and patterns used in the Squad codebase"
domain: "project-conventions"
confidence: "high"
source: "manual"
---

## Context
These conventions apply to all work on the Squad CLI tool (`create-squad`). Squad is a zero-dependency Node.js package that adds AI agent teams to any project. Understanding these patterns is essential before modifying any Squad source code.

## Patterns

### Zero Dependencies
Squad has zero runtime dependencies. Everything uses Node.js built-ins (`fs`, `path`, `os`, `child_process`). Do not add packages to `dependencies` in `package.json`. This is a hard constraint, not a preference.

### Node.js Built-in Test Runner
Tests use `node:test` and `node:assert/strict` — no test frameworks. Run with `npm test`. Test files live in `test/`. The test command is `node --test test/`.

### Error Handling — `fatal()` Pattern
All user-facing errors use the `fatal(msg)` function which prints a red `✗` prefix and exits with code 1. Never throw unhandled exceptions or print raw stack traces. The global `uncaughtException` handler calls `fatal()` as a safety net.

### ANSI Color Constants
Colors are defined as constants at the top of `index.js`: `GREEN`, `RED`, `DIM`, `BOLD`, `RESET`. Use these constants — do not inline ANSI escape codes.

### File Structure
- `.squad/` — Team state (user-owned, never overwritten by upgrades)
- `.squad/templates/` — Template files copied from `templates/` (Squad-owned, overwritten on upgrade)
- `.github/agents/squad.agent.md` — Coordinator prompt (Squad-owned, overwritten on upgrade)
- `templates/` — Source templates shipped with the npm package
- `.squad/skills/` — Team skills in SKILL.md format (user-owned)
- `.squad/decisions/inbox/` — Drop-box for parallel decision writes

### Windows Compatibility
Always use `path.join()` for file paths — never hardcode `/` or `\` separators. Squad must work on Windows, macOS, and Linux. All tests must pass on all platforms.

### Init Idempotency
The init flow uses a skip-if-exists pattern: if a file or directory already exists, skip it and report "already exists." Never overwrite user state during init. The upgrade flow overwrites only Squad-owned files.

### Copy Pattern
`copyRecursive(src, target)` handles both files and directories. It creates parent directories with `{ recursive: true }` and uses `fs.copyFileSync` for files.

## Examples

```javascript
// Error handling
function fatal(msg) {
  console.error(`${RED}✗${RESET} ${msg}`);
  process.exit(1);
}

// File path construction (Windows-safe)
const agentDest = path.join(dest, '.github', 'agents', 'squad.agent.md');

// Skip-if-exists pattern
if (!fs.existsSync(ceremoniesDest)) {
  fs.copyFileSync(ceremoniesSrc, ceremoniesDest);
  console.log(`${GREEN}✓${RESET} .squad/ceremonies.md`);
} else {
  console.log(`${DIM}ceremonies.md already exists — skipping${RESET}`);
}
```

## Anti-Patterns
- **Adding npm dependencies** — Squad is zero-dep. Use Node.js built-ins only.
- **Hardcoded path separators** — Never use `/` or `\` directly. Always `path.join()`.
- **Overwriting user state on init** — Init skips existing files. Only upgrade overwrites Squad-owned files.
- **Raw stack traces** — All errors go through `fatal()`. Users see clean messages, not stack traces.
- **Inline ANSI codes** — Use the color constants (`GREEN`, `RED`, `DIM`, `BOLD`, `RESET`).

<!-- chapter:end slug=squad-conventions -->

---

<!-- chapter:begin slug=squad-version-check position=9 -->

## 9. squad-version-check

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/squad-version-check/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/squad-version-check/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/squad-version-check.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

# SKILL: Squad CLI Internals — Version Stamping & Upgrade Mechanics

**Confidence:** medium
**Discovered by:** Data
**Date:** 2026-05-26
**Validated in:** Issue #1173 recon (bradygaster/squad)

---

## What This Skill Covers

Reusable knowledge about how `@bradygaster/squad-cli` stamps its version into `squad.agent.md`, how `squad upgrade` works, what it preserves vs. overwrites, and how to probe the npm registry for the latest version from a coordinator prompt.

---

## Package & Registry Facts

- **Package name:** `@bradygaster/squad-cli`
- **Registry:** npm (public)
- **CLI binary:** `squad` (registered via `package.json#bin.squad`)
- **Node version requirement:** Node ≥22.5.0 (ESM-only codebase)

---

## Version Stamping Mechanism

**Source file:** `dist/cli/core/version.js`

Three functions:

### `getPackageVersion()`
Walks up from the compiled JS file to find `package.json`. Returns `pkg.version`. Works from both `dist/cli/core/version.js` and a bundled root `cli.js`. Returns `'0.0.0'` as fallback if not found.

### `stampVersion(filePath, version)`
Mutates `squad.agent.md` in three places:
1. HTML comment: `<!-- version: {version} -->` (must be on the line immediately after frontmatter `---`)
2. Identity line: `- **Version:** {version}`
3. Greeting instruction: backtick-quoted `` `Squad v{version}` ``

**Called by:** both `init` and `upgrade` — after copying the template to the destination.

### `readInstalledVersion(filePath)`
Reads the stamped version back from `squad.agent.md`:
1. First tries HTML comment format: `/<!-- version: ([0-9.]+(?:-[a-z]+(?:\.\d+)?)?) -->/`
2. Falls back to old frontmatter format: `/^version:\s*"([^"]+)"/m`
3. Returns `'0.0.0'` on any error

---

## `squad upgrade` Behavior

**Source file:** `dist/cli/core/upgrade.js`

### What gets overwritten:
- `squad.agent.md` — full overwrite from template, then `stampVersion()`
- Files with `overwriteOnUpgrade: true` in `TEMPLATE_MANIFEST`: casting JSON files, template .md files, `copilot-instructions.md` (if @copilot enabled)
- GitHub Actions workflows — from `templates/workflows/`; non-npm projects get type-aware stubs
- Runs `runMigrations()` after file copy

### What is PRESERVED:
- `team.md`, `routing.md`, `decisions.md`, `ceremonies.md` (user-owned)
- `agents/*/history.md` (individual agent memory)
- `.squad/config.json` — **never touched**; `stateBackend` survives intact
- User-added files not in TEMPLATE_MANIFEST

### Self-upgrade path (`selfUpgradeCli()`):
Detects npm/pnpm/yarn via `npm_execpath` and `npm_config_user_agent`. Runs:
- npm: `npm install -g @bradygaster/squad-cli@latest`
- pnpm: `pnpm add -g @bradygaster/squad-cli@latest`
- yarn: `yarn global add @bradygaster/squad-cli@latest`
Use `@insider` tag for insider builds.

### `compareSemver(a, b)` utility (in upgrade.js):
Returns -1/0/1. Handles pre-release: strips pre-release for base comparison, then treats pre-release as less than release (e.g., `0.9.5-insider.1` < `0.9.5`). Can be ported directly if needed in prompt logic.

---

## `.squad/config.json` — What It Holds

```json
{
  "version": 1,
  "stateBackend": "worktree"
}
```

Other optional fields added by the coordinator at runtime:
- `defaultModel` — global model override for all agent spawns
- `agentModelOverrides.{agentName}` — per-agent model override

The file is read-only from the upgrade path's perspective. Only the coordinator writes to it (for model preferences).

---

## Version-Check Probe (npm Registry)

Use this one-liner from inside a coordinator prompt to fetch dist-tags:

```
npm view @bradygaster/squad-cli dist-tags --json
```

- Timeout: **5 seconds.** If no response within 5 seconds, abandon and show normal greeting.
- On success: extract `dist-tags[channel]` (e.g., `dist-tags["insider"]`).
- On any error (network failure, registry unreachable, parse error): show normal greeting.

---

## Upstream OS-Specific Cache

The CLI (`self-update.ts`) writes `latest` version info to an OS-specific path with a 24h TTL.

**One-liner to read the upstream cache:**
```
node -e "const p=require('path'),o=require('os');const b=process.env.APPDATA||(process.platform==='darwin'?p.join(o.homedir(),'Library','Application Support'):p.join(o.homedir(),'.config'));const f=p.join(b,'squad-cli','update-check.json');try{const d=JSON.parse(require('fs').readFileSync(f,'utf8'));const age=Date.now()-d.checkedAt;if(age<86400000)console.log(JSON.stringify(d));else console.log('STALE')}catch{console.log('MISS')}"
```

Output semantics:
- Valid JSON `{"latestVersion":"X.Y.Z","checkedAt":N}` → cache hit; use `latestVersion`
- `STALE` → cache expired (older than 24h); treat as no data
- `MISS` → cache missing or corrupt; treat as no data

**OS-specific cache path:**
- Windows: `%APPDATA%\squad-cli\update-check.json`
- Linux: `~/.config/squad-cli/update-check.json`
- macOS: `~/Library/Application Support/squad-cli/update-check.json`

---

## Repo-Local Cache Convention: `.squad/.cache/version-check.json`

Used by coordinator for `insider`/`preview` channels (the upstream cache only stores `latest`).

**Schema:**
```json
{
  "checkedAt": "2026-05-26T14:13:28.492Z",
  "currentVersion": "0.9.6-insider.2",
  "channel": "insider",
  "channelVersion": "0.9.7-insider.1"
}
```

**TTL:** 24 hours from `checkedAt`.
**Gitignore:** `.squad/.cache/` is listed in `.gitignore` — cache files are never committed.

---

## Key File Paths (installed CLI)

| Purpose | Path |
|---|---|
| Version utilities | `dist/cli/core/version.js` |
| Upgrade logic | `dist/cli/core/upgrade.js` |
| Init logic | `dist/cli/core/init.js` |
| Template manifest | `dist/cli/core/templates.js` |
| Copilot install helper | `dist/cli/copilot-install.js` |
| squad.agent.md template | `templates/squad.agent.md.template` |
| Session init reference | `templates/session-init-reference.md` |
| All templates | `templates/` |

<!-- chapter:end slug=squad-version-check -->

---

<!-- chapter:begin slug=test-discipline position=10 -->

## 10. test-discipline

- **Source:** https://github.com/microsoft/waza/blob/main/.copilot/skills/test-discipline/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.copilot/skills/test-discipline/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/test-discipline.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "test-discipline"
description: "Update tests when changing APIs — no exceptions"
domain: "quality"
confidence: "high"
source: "earned (Fenster/Hockney incident, test assertion sync violations)"
---

## Context

When APIs or public interfaces change, tests must be updated in the same commit. When test assertions reference file counts or expected arrays, they must be kept in sync with disk reality. Stale tests block CI for other contributors.

## Patterns

- **API changes → test updates (same commit):** If you change a function signature, public interface, or exported API, update the corresponding tests before committing
- **Test assertions → disk reality:** When test files contain expected counts (e.g., `EXPECTED_FEATURES`, `EXPECTED_SCENARIOS`), they must match the actual files on disk
- **Add files → update assertions:** When adding docs pages, features, or any counted resource, update the test assertion array in the same commit
- **CI failures → check assertions first:** Before debugging complex failures, verify test assertion arrays match filesystem state

## Examples

✓ **Correct:**
- Changed auth API signature → updated auth.test.ts in same commit
- Added `distributed-mesh.md` to features/ → added `'distributed-mesh'` to EXPECTED_FEATURES array
- Deleted two scenario files → removed entries from EXPECTED_SCENARIOS

✗ **Incorrect:**
- Changed spawn parameters → committed without updating casting.test.ts (CI breaks for next person)
- Added `built-in-roles.md` → left EXPECTED_FEATURES at old count (PR blocked)
- Test says "expected 7 files" but disk has 25 (assertion staleness)

## Anti-Patterns

- Committing API changes without test updates ("I'll fix tests later")
- Treating test assertion arrays as static (they evolve with content)
- Assuming CI passing means coverage is correct (stale assertions can pass while being wrong)
- Leaving gaps for other agents to discover

<!-- chapter:end slug=test-discipline -->

---

<!-- chapter:begin slug=squad-templates position=11 -->

## 11. {skill-name}

- **Source:** https://github.com/microsoft/waza/blob/main/.squad-templates/skill.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/skill.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/squad-templates.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

Bundled files (31), referenced from this skill's directory:
  - `casting-history.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/casting-history.json
  - `casting-policy.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/casting-policy.json
  - `casting-registry.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/casting-registry.json
  - `ceremonies.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/ceremonies.md
  - `charter.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/charter.md
  - `constraint-tracking.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/constraint-tracking.md
  - `copilot-instructions.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/copilot-instructions.md
  - `history.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/history.md
  - `identity/now.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/identity/now.md
  - `identity/wisdom.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/identity/wisdom.md
  - `mcp-config.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/mcp-config.md
  - `multi-agent-format.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/multi-agent-format.md
  - `orchestration-log.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/orchestration-log.md
  - `plugin-marketplace.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/plugin-marketplace.md
  - `raw-agent-output.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/raw-agent-output.md
  - `roster.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/roster.md
  - `routing.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/routing.md
  - `run-output.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/run-output.md
  - `scribe-charter.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/scribe-charter.md
  - `workflows/squad-ci.yml` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/workflows/squad-ci.yml
  - `workflows/squad-docs.yml` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/workflows/squad-docs.yml
  - `workflows/squad-heartbeat.yml` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/workflows/squad-heartbeat.yml
  - `workflows/squad-insider-release.yml` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/workflows/squad-insider-release.yml
  - `workflows/squad-issue-assign.yml` — https://raw.githubusercontent.com/microsoft/waza/main/.squad-templates/workflows/squad-issue-assign.yml
  - …and 7 more, listed in https://skillsdocs.com/api/v1/books/microsoft/waza/skills/squad-templates

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "{skill-name}"
description: "{what this skill teaches agents}"
domain: "{e.g., testing, api-design, error-handling}"
confidence: "low|medium|high"
source: "{how this was learned: manual, observed, earned}"
tools:
  # Optional — declare MCP tools relevant to this skill's patterns
  # - name: "{tool-name}"
  #   description: "{what this tool does}"
  #   when: "{when to use this tool}"
---

## Context
{When and why this skill applies}

## Patterns
{Specific patterns, conventions, or approaches}

## Examples
{Code examples or references}

## Anti-Patterns
{What to avoid}

<!-- chapter:end slug=squad-templates -->

---

<!-- chapter:begin slug=templates position=12 -->

## 12. {skill-name}

- **Source:** https://github.com/microsoft/waza/blob/main/.squad/templates/skill.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/skill.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/templates.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

Bundled files (49), referenced from this skill's directory:
  - `after-agent-reference.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/after-agent-reference.md
  - `casting-history.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/casting-history.json
  - `casting-policy.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/casting-policy.json
  - `casting-reference.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/casting-reference.md
  - `casting-registry.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/casting-registry.json
  - `casting/Futurama.json` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/casting/Futurama.json
  - `ceremonies.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/ceremonies.md
  - `ceremony-reference.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/ceremony-reference.md
  - `charter.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/charter.md
  - `client-compatibility-reference.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/client-compatibility-reference.md
  - `constraint-tracking.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/constraint-tracking.md
  - `cooperative-rate-limiting.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/cooperative-rate-limiting.md
  - `copilot-agent.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/copilot-agent.md
  - `copilot-instructions.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/copilot-instructions.md
  - `fact-checker-charter.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/fact-checker-charter.md
  - `history.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/history.md
  - `identity/now.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/identity/now.md
  - `identity/wisdom.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/identity/wisdom.md
  - `issue-lifecycle.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/issue-lifecycle.md
  - `keda-scaler.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/keda-scaler.md
  - `loop.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/loop.md
  - `machine-capabilities.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/machine-capabilities.md
  - `mcp-config.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/mcp-config.md
  - `model-selection-reference.md` — https://raw.githubusercontent.com/microsoft/waza/main/.squad/templates/model-selection-reference.md
  - …and 25 more, listed in https://skillsdocs.com/api/v1/books/microsoft/waza/skills/templates

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: "{skill-name}"
description: "{what this skill teaches agents}"
domain: "{e.g., testing, api-design, error-handling}"
confidence: "low|medium|high"
source: "{how this was learned: manual, observed, earned}"
tools:
  # Optional — declare MCP tools relevant to this skill's patterns
  # - name: "{tool-name}"
  #   description: "{what this tool does}"
  #   when: "{when to use this tool}"
---

## Context
{When and why this skill applies}

## Patterns
{Specific patterns, conventions, or approaches}

## Examples
{Code examples or references}

## Anti-Patterns
{What to avoid}

<!-- chapter:end slug=templates -->

---

<!-- chapter:begin slug=code-explainer position=13 -->

## 13. code-explainer

- **Source:** https://github.com/microsoft/waza/blob/main/skills/code-explainer/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/skills/code-explainer/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/code-explainer.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: code-explainer
description: Explains code snippets in plain English, breaking down what the code does step by step. Perfect for learning, code reviews, or documentation.
---

# Code Explainer Skill

You are a code explanation expert. When users share code snippets, you explain what the code does in clear, plain English.

## Instructions

1. **Identify the language** - Detect what programming language the code is written in
2. **Break it down** - Explain the code step by step
3. **Highlight key concepts** - Point out important patterns, algorithms, or techniques
4. **Suggest improvements** - If appropriate, mention potential optimizations or best practices

## Output Format

Structure your explanation as:

1. **Overview**: A one-sentence summary of what the code does
2. **Step-by-Step**: Numbered explanation of each significant part
3. **Key Concepts**: Important programming concepts used
4. **Suggestions** (optional): Improvements or alternatives

## Examples

### Example 1: Simple Function

**User**: Explain this code:
```python
def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n - 1)
```

**Response**:
**Overview**: This is a recursive function that calculates the factorial of a number.

**Step-by-Step**:
1. The function `factorial` takes a single parameter `n`
2. Base case: If `n` is 0 or 1, return 1 (since 0! = 1! = 1)
3. Recursive case: Multiply `n` by the factorial of `n-1`
4. This continues until reaching the base case

**Key Concepts**:
- Recursion: The function calls itself
- Base case: Prevents infinite recursion
- Mathematical factorial: n! = n × (n-1) × ... × 1

### Example 2: List Comprehension

**User**: What does this do?
```python
squares = [x**2 for x in range(10) if x % 2 == 0]
```

**Response**:
**Overview**: Creates a list of squares of even numbers from 0 to 9.

**Step-by-Step**:
1. `range(10)` generates numbers 0-9
2. `if x % 2 == 0` filters to only even numbers (0, 2, 4, 6, 8)
3. `x**2` squares each filtered number
4. Result: `[0, 4, 16, 36, 64]`

**Key Concepts**:
- List comprehension: Compact way to create lists
- Filtering: Conditional inclusion with `if`
- Modulo operator: `%` checks divisibility

## Behavior Guidelines

- Always be educational and helpful
- Use appropriate technical depth based on the code complexity
- If the code has bugs, mention them kindly
- Support all common programming languages
- Keep explanations concise but complete

<!-- chapter:end slug=code-explainer -->

---

<!-- chapter:begin slug=waza-interactive position=14 -->

## 14. waza-interactive

- **Source:** https://github.com/microsoft/waza/blob/main/skills/waza-interactive/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/skills/waza-interactive/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/waza-interactive.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

Bundled files (1), referenced from this skill's directory:
  - `tests/eval.yaml` — https://raw.githubusercontent.com/microsoft/waza/main/skills/waza-interactive/tests/eval.yaml

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: waza-interactive
description: "Interactive workflow partner for creating, testing, and improving AI agent skills with waza. USE FOR: run my evals, check my skill, compare models, create eval suite, debug failing tests, is my skill ready, ship readiness, interpret results, improve score. DO NOT USE FOR: general coding, non-skill work, writing skill content (use skill-authoring), improving frontmatter only (use sensei)."
---

# Waza Interactive

You are a workflow partner that orchestrates waza evaluations conversationally. Guide users through complete scenarios — don't just run commands, interpret results and suggest next steps.

## Available MCP Tools

Call these tools to execute waza operations:

| Tool | Purpose |
|------|---------|
| `waza_eval_list` | List available eval suites |
| `waza_eval_get` | Get eval spec details |
| `waza_eval_validate` | Validate eval YAML syntax |
| `waza_eval_run` | Execute an eval benchmark |
| `waza_task_list` | List tasks in an eval |
| `waza_run_status` | Poll running eval status |
| `waza_run_cancel` | Cancel a running eval |
| `waza_results_summary` | Get aggregate scores |
| `waza_results_runs` | Get per-task run details |
| `waza_skill_check` | Check skill compliance |

## Scenario 1: Create a New Eval

When user wants to create an eval suite for their skill:

1. Ask which skill to evaluate — get the skill name and path
2. Call `waza_eval_list` to check for existing evals for this skill
3. If none exist, run `waza init <directory>` via terminal to scaffold
4. Explain the generated `eval.yaml` structure — name, skill, executor, tasks
5. Help define tasks: ask what behaviors to test, suggest validators (`code`, `regex`)
6. For each task, help write the prompt and expected output
7. Call `waza_eval_validate` to confirm the YAML is valid
8. Suggest running with `waza_eval_run` to verify the first task passes

**Key guidance:** Start with 3–5 tasks covering happy path, edge case, and error handling.

## Scenario 2: Run and Interpret Results

When user wants to run evals and understand scores:

1. Call `waza_eval_run` with the eval spec path and context dir
2. Poll `waza_run_status` until complete (check every 10s)
3. Call `waza_results_summary` to get aggregate scores
4. Interpret the results for the user:
   - **Pass rate** — percentage of tasks that passed all validators
   - **Weighted score** — 0.0–1.0 aggregate across all tasks
   - **Duration** — total and per-task execution time
5. If pass rate < 80%, identify which tasks failed and why
6. Call `waza_results_runs` for per-task details on failures
7. Suggest specific improvements: prompt rewording, validator tuning, fixture updates

**Thresholds:** ≥90% pass rate = strong, 70–89% = needs work, <70% = significant issues.

## Scenario 3: Compare Models

When user wants to compare model performance:

1. Ask which models to compare (e.g., gpt-4o vs claude-sonnet-4)
2. Call `waza_eval_run` with model A — save results
3. Call `waza_eval_run` with model B — save results
4. Compare results side by side:
   - Per-task pass/fail differences
   - Score deltas (which model scores higher on which tasks)
   - Duration differences (speed vs quality tradeoff)
5. Provide a recommendation: which model is better for this skill and why
6. Suggest next steps: try a third model, tune prompts for the weaker model, or adjust validators

**Guidance:** Run each model 2–3 times to account for variance before drawing conclusions.

## Scenario 4: Debug a Failing Skill

When user's skill is failing evals or behaving unexpectedly:

1. Call `waza_skill_check` to verify skill compliance (frontmatter, triggers, token count)
2. If compliance issues found, fix those first — they affect routing
3. Call `waza_eval_run` with `--verbose` and `--transcript-dir` flags
4. Call `waza_results_runs` to get per-task failure details
5. Analyze failure patterns:
   - **All tasks fail** → prompt or fixture issue, check skill instructions
   - **Some tasks fail** → specific edge cases, review failed task prompts
   - **Validator failures** → regex too strict, code validator language mismatch
6. Suggest targeted fixes based on the pattern
7. Re-run with `waza_eval_run` to verify the fix

## Scenario 5: Ship Readiness Check

When user asks "is my skill ready?" or wants a pre-ship checklist:

1. Call `waza_skill_check` — verify compliance score ≥ medium-high
2. Call `waza_eval_validate` — confirm eval YAML is valid
3. Call `waza_eval_run` — execute full eval suite
4. Call `waza_results_summary` — check aggregate scores
5. Render the readiness verdict:

```
SHIP READINESS CHECKLIST:
☐ Skill compliance: [score] (need: medium-high+)
☐ Eval YAML valid: [yes/no]
☐ Pass rate: [X]% (need: ≥90%)
☐ Weighted score: [X.XX] (need: ≥0.85)
☐ No task timeouts
☐ Consistent across 2+ runs

VERDICT: [READY / NOT READY — fix items marked ✗]
```

6. If NOT READY, route to the appropriate scenario (Scenario 4 for failures, Scenario 1 for missing evals)

## Conversation Style

- Always explain *why* before *what* — context before commands
- After every tool call, interpret the result in plain language
- When something fails, diagnose before suggesting fixes
- Offer the next logical step — don't wait to be asked
- Use the checklist format for multi-step validations

<!-- chapter:end slug=waza-interactive -->

---

<!-- chapter:begin slug=waza position=15 -->

## 15. waza

- **Source:** https://github.com/microsoft/waza/blob/main/skills/waza/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/skills/waza/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/waza.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: waza
description: "**WORKFLOW SKILL** - Evaluate AI agent skills using structured benchmarks with YAML specs, fixture isolation, and pluggable validators. USE FOR: run waza, waza help, run eval, run benchmark, evaluate skill, test agent, generate eval suite, init eval, compare results, score agent, agent evaluation, skill testing, cross-model comparison. DO NOT USE FOR: improving skill frontmatter (use waza dev), creating new skills from scratch (use skill-creator), token counting or budget checks (use waza tokens). INVOKES: Copilot SDK executor, mock engine, code/regex validators. FOR SINGLE OPERATIONS: use waza run directly for a single benchmark."
---

# Waza

> "The way of technique — measure, refine, master."

A Go CLI tool for evaluating AI agent skills through structured benchmarks. Define test cases in YAML, run them against agent engines, and validate results with pluggable scoring validators.

## Help

When user says "waza help" or asks how to use waza:

```
╔══════════════════════════════════════════════════════════════════╗
║  WAZA - CLI Tool for Evaluating Agent Skills                     ║
╠══════════════════════════════════════════════════════════════════╣
║                                                                  ║
║  COMMANDS:                                                       ║
║    waza run <eval.yaml>        # Run an evaluation benchmark     ║
║    waza init [directory]       # Initialize a new eval suite     ║
║    waza generate <SKILL.md>    # Generate eval from SKILL.md     ║
║    waza compare <r1> <r2> ...  # Compare result files            ║
║    waza dev [skill-path]       # Improve SKILL.md compliance     ║
║                                                                  ║
║  RUN FLAGS:                                                      ║
║    --context-dir, -c   Fixtures directory (default: ./fixtures)  ║
║    --output, -o        Save results JSON to file                 ║
║    --verbose, -v       Verbose output                            ║
║    --task, -t          Filter tasks by name (repeatable)         ║
║    --parallel, -p      Run tasks in parallel                     ║
║    --workers, -w       Number of parallel workers                ║
║    --transcript-dir    Save per-task transcripts                 ║
║                                                                  ║
║  COMPARE FLAGS:                                                  ║
║    --format, -f        Output format: table or json              ║
║                                                                  ║
║  GENERATE FLAGS:                                                 ║
║    --output-dir, -d    Output directory for generated files      ║
║                                                                  ║
║  DEV FLAGS:                                                      ║
║    --target            Adherence level: low|medium|high          ║
║    --max-iterations    Max improvement iterations (default: 5)   ║
║    --auto              Auto-apply without prompting              ║
║                                                                  ║
║  WORKFLOW:                                                       ║
║    1. waza init my-eval        # Scaffold eval suite             ║
║    2. Edit eval.yaml + tasks   # Define test cases               ║
║    3. waza run eval.yaml -v    # Execute benchmark               ║
║    4. waza compare a.json b.json  # Cross-model comparison       ║
║                                                                  ║
║  FIXTURE ISOLATION:                                              ║
║    Each task gets a fresh temp workspace with fixtures copied    ║
║    in. Original fixtures are never modified.                     ║
║                                                                  ║
╚══════════════════════════════════════════════════════════════════╝
```

## Commands

### `waza run`

Run an evaluation benchmark from a YAML spec file.

```bash
# Run with default mock engine
waza run path/to/eval.yaml --context-dir path/to/fixtures

# Verbose output with results saved
waza run eval.yaml -c ./fixtures -v -o results.json

# Filter to specific tasks
waza run eval.yaml -t "task-name-1" -t "task-name-2"

# Parallel execution
waza run eval.yaml --parallel --workers 4

# Save per-task transcripts
waza run eval.yaml --transcript-dir ./transcripts
```

### `waza init`

Initialize a new evaluation suite with a compliant directory structure.

```bash
# Initialize in current directory
waza init

# Initialize in a named directory
waza init my-eval-suite
```

Creates: `eval.yaml`, `tasks/` with example task, `fixtures/` with example fixture.

### `waza generate`

Generate an eval suite from an existing SKILL.md file.

```bash
# Generate eval from SKILL.md
waza generate path/to/SKILL.md

# Specify output directory
waza generate SKILL.md --output-dir ./my-eval
```

Parses YAML frontmatter (name, description) and creates eval.yaml, starter tasks, and fixtures.

### `waza compare`

Compare results from multiple evaluation runs side by side.

```bash
# Compare two result files
waza compare run1.json run2.json

# Compare three or more
waza compare gpt4.json claude.json gemini.json

# JSON output
waza compare run1.json run2.json --format json
```

Shows per-task score deltas, pass rate differences, and aggregate statistics.

### `waza dev`

Iteratively improve SKILL.md frontmatter compliance with automated scoring.

```bash
# Score current skill and suggest improvements
waza dev skills/my-skill

# Target high compliance level
waza dev skills/my-skill --target high

# Auto-apply improvements without prompts
waza dev skills/my-skill --target medium --auto --max-iterations 3
```

**Compliance Levels:**
- **Low** (< 150 chars or no triggers) — Minimal description
- **Medium** (150+ chars, has triggers) — Basic trigger coverage
- **Medium-High** (+ anti-triggers) — Routing clarity improved
- **High** (+ routing markers like INVOKES/FOR SINGLE OPERATIONS) — Full compliance

**Scoring Checks:**
- Description length (150+ chars required, 1024 max)
- Trigger phrases (USE FOR: patterns)
- Anti-trigger phrases (DO NOT USE FOR: patterns)
- Routing clarity markers (**WORKFLOW SKILL**, INVOKES:, etc.)
- Token budget (500 soft limit, 5000 hard limit)

**Coming Soon:** Trigger accuracy tests (#36), `--skip-integration` (#37), `--fast` (#38), improvement suggestions engine (#34).

## Evaluation Spec Format

```yaml
name: my-eval
skill: my-skill
version: "1.0"
executor: mock          # or copilot-sdk
tasks:
  - id: task-1
    name: "Describe the task"
    prompt: "Your prompt to the agent"
    expected: "Expected behavior"
    validators:
      - type: code
        config:
          language: go
      - type: text
        config:
          pattern: "expected pattern"
```

## Engines

| Engine | Use | Description |
|--------|-----|-------------|
| `mock` | Testing | Returns canned responses for validator development |
| `copilot-sdk` | Production | Executes via Copilot CLI SDK |

## Validators

| Validator | What it checks |
|-----------|---------------|
| `code` | Code compiles / passes syntax check |
| `regex` | Output matches regex pattern |

## Configuration

| Setting | Flag | Default |
|---------|------|---------|
| Fixtures dir | `--context-dir` | `./fixtures` |
| Output file | `--output` | (none) |
| Verbose | `--verbose` | `false` |
| Parallel | `--parallel` | `false` |
| Workers | `--workers` | CPU count |
| Transcript dir | `--transcript-dir` | (none) |

## Scoring Quick Reference

Each task produces an `EvaluationOutcome` with:

| Field | Description |
|-------|-------------|
| `score` | 0.0–1.0 normalized score |
| `pass` | Boolean pass/fail |
| `validator_results` | Per-validator details |
| `duration` | Execution time |

<!-- chapter:end slug=waza -->

---

<!-- chapter:begin slug=waza-runner position=16 -->

## 16. waza-runner

- **Source:** https://github.com/microsoft/waza/blob/main/waza-runner/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/waza-runner/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/waza-runner.md
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

Bundled files (1), referenced from this skill's directory:
  - `references/EVAL-SPEC.md` — https://raw.githubusercontent.com/microsoft/waza/main/waza-runner/references/EVAL-SPEC.md

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: waza-runner
description: |
  Run evaluations on Agent Skills to measure their effectiveness. 
  USE FOR: "run skill evals", "evaluate my skill", "test skill quality", 
  "check skill triggers", "skill compliance check", "measure skill performance",
  "run evals on [skill-name]", "grade skill execution".
  DO NOT USE FOR: writing skills (use skill-authoring), improving frontmatter 
  (use sensei), or general testing unrelated to skills.
metadata:
  author: spboyer
  version: "1.0"
---

# Skill Eval Runner

> Evaluate Agent Skills like you evaluate AI Agents

This skill runs evaluations on other skills to measure their effectiveness using the same patterns that power AI agent evaluations.

## When to Use

- Running quality evaluations on a skill
- Testing if a skill triggers on correct prompts
- Measuring skill behavior quality
- Generating eval reports for CI/CD

## Commands

### Run Evals
```
Run evals on <skill-name>
```

### Initialize Eval Suite
```
Create evals for <skill-name>
```

### Generate Report
```
Generate eval report for <skill-name>
```

## Workflow

1. **Check for Eval Suite**: Look for `eval.yaml` in the skill directory
2. **Load Tasks**: Parse task definitions from `tasks/*.yaml`
3. **Execute**: Run each task through the configured graders
4. **Report**: Output results in JSON or Markdown format

## Metrics Measured

| Metric | Description | Default Threshold |
|--------|-------------|-------------------|
| Task Completion | Did the skill accomplish the goal? | 80% |
| Trigger Accuracy | Was skill invoked on correct prompts? | 90% |
| Behavior Quality | Tool calls, efficiency, reasoning | 70% |

## Grader Types

- **Code Graders**: Deterministic assertions, regex matching
- **LLM Graders**: Model-as-judge with configurable rubrics
- **Human Graders**: Manual review workflow

## Example Usage

### Running Evals
```bash
# From CLI
waza run ./my-skill/eval.yaml

# Output to file
waza run ./my-skill/eval.yaml -o results.json
```

### Interpreting Results
```json
{
  "summary": {
    "pass_rate": 0.85,
    "composite_score": 0.82
  },
  "metrics": {
    "task_completion": { "score": 0.9, "passed": true },
    "trigger_accuracy": { "score": 0.95, "passed": true }
  }
}
```

## References

- [Eval Specification](references/EVAL-SPEC.md) - Full eval.yaml schema
- [Writing Tasks](references/WRITING-TASKS.md) - Task definition guide
- [Grader Reference](references/GRADERS.md) - Available graders

<!-- chapter:end slug=waza-runner -->

---

## Part: Credited skills

---

<!-- chapter:begin slug=azd-publish position=17 -->

## 17. azd-publish

- **Source:** https://github.com/microsoft/waza/blob/main/.github/skills/azd-publish/SKILL.md
- **Raw:** https://raw.githubusercontent.com/microsoft/waza/main/.github/skills/azd-publish/SKILL.md
- **Markdown:** https://skillsdocs.com/microsoft/waza/azd-publish.md
- **Origin:** Credited — installed into this repository, not published from it.
- **Licence:** MIT — https://spdx.org/licenses/MIT.html

<!-- Verbatim upstream SKILL.md follows, YAML frontmatter included. -->

---
name: azd-publish
description: |
  Prepare and publish a new version of the waza azd extension.
  USE FOR: "publish extension", "release new version", "bump version",
  "prepare release", "update changelog", "azd publish", "new release",
  "version bump", "cut a release".
  DO NOT USE FOR: running evals (use waza), writing skills (use skill-authoring),
  CI/CD pipeline changes (edit workflow files directly).
metadata:
  author: spboyer
  version: "1.0"
---

# azd Extension Publish

> Automate version bumps, changelog updates, and PR creation for waza azd extension releases.

## When to Use

- Preparing a new release of the waza azd extension
- Bumping the version number (major, minor, or patch)
- Updating the changelog with changes since last release
- Creating a release PR for review

## Workflow

Follow these steps **in order**. Ask the user for input at each decision point.

### Step 1: Gather Changes and Update Changelog

Get the current version from `version.txt` and `extension.yaml`, then collect commits since the last release:

```bash
cat version.txt

# Find the latest azd extension version tags
git tag --list 'azd-ext-microsoft-azd-waza_*' --sort=-v:refname | head -5

# Get commits since last azd extension tag
last_tag=$(git tag --list 'azd-ext-microsoft-azd-waza_*' --sort=-v:refname | head -1)
git log "${last_tag}..HEAD" --oneline --no-decorate
```

If `version.txt` and `extension.yaml` differ, flag it to the user before proceeding.

Summarize the changes grouped by type:
- **Added** — `feat:` commits
- **Fixed** — `fix:` commits
- **Changed** — `refactor:`, `chore:`, `docs:` commits
- **Removed** — any removal-related commits

Present the summary to the user for review.

Then update `CHANGELOG.md`. The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format.

Perform these updates (using a placeholder version `X.Y.Z` — the actual version is determined in Step 2):

1. **Move Unreleased content**: Move any items currently under `## [Unreleased]` into a staging area. If `[Unreleased]` is empty, populate from the git log summary gathered above.

2. **Populate from commits**: Prepare entries grouped under `### Added`, `### Fixed`, `### Changed` as appropriate based on the commits gathered above.

Hold these changelog entries — the new version section header and comparison links will be finalized after the version is determined in Step 2.

### Step 2: Determine Version

Based on the changes gathered in Step 1, **recommend** a version bump type using standard semver semantics:

- **major** — Breaking changes, removals of public API (`feat!:`, `BREAKING CHANGE:`) → `(MAJOR+1).0.0`
- **minor** — New features, backward compatible (`feat:`) → `MAJOR.(MINOR+1).0`
- **patch** — Bug fixes, docs, refactors, chores (`fix:`, `docs:`, `refactor:`, `chore:`) → `MAJOR.MINOR.(PATCH+1)`

Present the recommendation with rationale (e.g., "I see 3 `feat:` commits and no breaking changes — recommending a **minor** bump").

**ASK THE USER** to confirm the recommended bump or choose a different one.

Compute the new version and confirm with the user before proceeding.

Then finalize the changelog:

1. **Create new version section**: Insert a new section below `## [Unreleased]` with today's date:
   ```markdown
   ## [X.Y.Z] - YYYY-MM-DD
   ```

2. **Add the prepared entries** from Step 1 under the new version section.

3. **Update comparison links** at the bottom of the file:
   ```markdown
   [Unreleased]: https://github.com/microsoft/waza/compare/azd-ext-microsoft-azd-waza_X.Y.Z...HEAD
   [X.Y.Z]: https://github.com/microsoft/waza/compare/azd-ext-microsoft-azd-waza_PREVIOUS...azd-ext-microsoft-azd-waza_X.Y.Z
   ```

4. **Clear the Unreleased section**: Leave `## [Unreleased]` with empty subsections or blank.

### Step 3: Update Version Files

Update these files with the new version:

1. **`version.txt`** — Replace contents with new version string
2. **`extension.yaml`** — Update the `version:` field

### Step 4: Review Changes

Show the user a summary of all changes made:
- New version number
- Files modified: `version.txt`, `extension.yaml`, `CHANGELOG.md`
- Show the diff with `git diff`

### Step 5: Ask About PR Creation

**ASK THE USER**: Should I create a PR with these changes?

If **yes**:

1. Create a feature branch:
   ```bash
   git checkout -b release/v{VERSION}
   ```

2. Stage and commit all changes:
   ```bash
   git add version.txt extension.yaml CHANGELOG.md
   git commit -m "chore: Prepare release v{VERSION}"
   ```

3. Push the branch:
   ```bash
   git push origin release/v{VERSION}
   ```

4. Create a PR using the GitHub CLI:
   ```bash
   gh pr create \
     --title "Release v{VERSION}" \
     --body "## Release v{VERSION}

   ### Changes
   {changelog entries for this version}

   ### Checklist
   - [ ] Version bumped in version.txt and extension.yaml
   - [ ] CHANGELOG.md updated
   - [ ] CI passes
   - [ ] Ready to publish via 'Publish azd Extension' workflow" \
     --base main \
     --head release/v{VERSION}
   ```

If **no**:
- Leave the changes uncommitted in the working tree
- Inform the user they can review and commit manually

## File Reference

| File | Purpose | What Gets Updated |
|------|---------|-------------------|
| `version.txt` | Single source of version truth | New semver version string |
| `extension.yaml` | azd extension manifest | `version:` field |
| `CHANGELOG.md` | Human-readable change history | New version section with entries |

## Important Notes

- Always use **conventional commit** prefixes (`feat:`, `fix:`, `chore:`, `docs:`, `refactor:`) when interpreting git history
- The changelog format must follow [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
- Version numbering must follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
- The PR branch naming convention is `release/v{VERSION}`
- After the PR is merged, the user should trigger the **Publish azd Extension** workflow (`azd-ext-release.yml`) to build, pack, and publish the extension

<!-- chapter:end slug=azd-publish -->