README

behavior-baseline.json is the recorded benchmark that intentional skill changes must not regress silently.

Do not add trigger evals for this skill

It was found that the run_eval.py‘s methodology incorrectly failed to detect skill triggers due to requiring immediate skill execution rather than context exploration, which models almost always do regardless of description changes.

This skill’s load-bearing invocation paths are the doc-parity Stop hook and the ai-review workflow’s documentation pass. Natural-language triggering is a nice-to-have fallback, not the critical path.

Eval cases

Behavior cases run against the Seeder subsystem of bitwarden/server (opens in a new tab) (util/Seeder/, util/SeederApi/, util/SeederUtility/), chosen because its code-to-docs mapping is known.

These evals are executed against a pinned server ref, d6c84a7562cc6b464de910dbf829690885500137. To increase stability in testing, evaluate against that reference point.

Each case is constructed so a run that skips a documented scope, stays silent instead of attesting, token-edits its way past the gate, or mishandles a specific outcome type (Update, Attest, Dismiss) fails a named expectation. Behavior cases assume the run happens inside a bitwarden/server checkout so the ground-truth docs are readable.

Case 10 exercises the review context (out-of-repo discovery against the contributing-docs corpus); all other cases are session cases. Cases involve live edits to a scratch checkout, so run them against a disposable clone or a git worktree, never a working tree you care about.

Each case’s expectations array is the pass criterion — every expectation is graded independently. Denominators differ per case because they count expectations, not runs.

Running

Run with /skill-creator:skill-creator in benchmark mode (with-skill vs. without-skill) with a config-blind grader. Install the runner from the claude-plugins-official marketplace if it is not already present:

The AI Review Guidelines want at least three iterations per case per configuration so pass rate and variance are both captured.

Regression check against the current baseline compares the aggregate summary and per-run pass rates; timestamps, models, and per-run evidence prose vary on every execution and are not signal:

An empty diff on that projection means no regression on the graded data. When a change is intentional and the new numbers are the new desired state, replace the baseline with the new results in the same PR as the skill change.

Updating the test surface

Update behavior-eval.json when the test surface needs to evolve: a new load-bearing rule in the skill, a Seeder doc restructure that invalidates a case’s ground truth, or an expectation that turned out to be ambiguous. The eval set encodes intent, not implementation — when a run fails, fix the skill, not the case, unless the case itself is wrong.