Setting the file. One moment.
Skill 11 · Architecting Solutions
Subchapter 11.3
evals/README.mdMarkdown1 KBView on GitHub
Behavior test cases for the architecting-solutions skill, in the skill-creator schema.
behavior-eval.json holds nine cases targeting the Bitwarden-specific parts of the skill.
Each case’s are the pass criteria. Denominators differ per case because they count expectations, not runs — every expectation is graded independently for both configurations.
expectationsCases are advice-only — they grade the design the skill produces and run no live edits, commits, or PRs, so re-runs are mutation-safe.
Run with /skill-creator:skill-creator in Benchmark mode (with-skill vs. without-skill) with a config-blind grader. Any change to SKILL.md should be paired with a re-run and a refresh of behavior-baseline.json; the baseline is what future comparisons diff against.
behavior-baseline.json records the pass/fail rate per case, keyed by model + effort. Regression check:
diff <(jq -S . behavior-baseline.json) <(jq -S . result.json)An empty diff means no regression. When a change is intentional and the new numbers are the new desired state, replace behavior-baseline.json with the new results in the same PR as the skill change.